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General Guidelines Overview 


Welcome to the Search Quality Rating Program! 
Аз a Search Quality Rater, you will work on many different types of rating projects. The General Guidelines primarily 
cover Page Quality (PQ) rating and Needs Met (NM) rating; however, the concepts are also important for many other 
types of rating tasks. 


For brevity, we refer to “Search Quality Raters” as “raters? in these guidelines. 





‘Should | buy an iPhone? 


Google 


me D 











Is this page trustworthy? 


Buy cheap iPhones for cheap - BuyCheaplphones.123 
wnwibujchespphones.122/buy cheap cheaper 

Browse and buy iPhones on the webs number #1 cheap IPhone marketplace. 
Register for FREE! Get cheap Phones here 


Copyright 2021 5 


Introduction to Search Quality Rating 


0.0 The Search Experience 


The World Wide Web is a vast collection of online information and content. Internet search engines provide a powerful 
way to explore this online universe. There are many ways people search: people may type words into a search box in a 
browser, speak to a mobile phone or assistant device, use search engine autocomplete features, etc. 


People search the Internet for a variety of purposes, ranging from accomplishing a quick task to researching a topic in 
depth. A search may be part of a long-term project, such as a home remodel or vacation planning. A search may be 
done when someone is bored and looking for entertainment, such as a search for [funny videos]. A search may be a 
single question asked during а critical moment of a person's Ife, such as [what are the symptoms of a heart attack?]. 


‘Search engines exist to help people find what they are looking for. To do that, search engines must provide a diverse set 
of helpful, high quality search results, presented in the most helpful order. 





Different types of searches need very diferent types of search results. Medical search results should be high quality, 
authoritative, and trustworthy. Search results for "cute baby animal pictures" should be adorable. Search results for a 

specific website or webpage should have that desired result at the top. Searches that have many possible meanings or 
involve many perspectives need a diverse sat of results that reflect the natural diversity of meanings and points of view. 


People all over the world use search engines; therefore, diversity in search results is essential to satisfy the diversity of 
people who use search. For example, searches about groups of people should retum helpful results that represent a 
diversity of demographic backgrounds and cultures. 


Finally, search results should help people. Search results should provide authoritative and trustworthy information, not 
lead people astray with misleading content. Search results should allow people to find what they're looking for, not 
surprise people with unpleasant, upsetting, offensive, or disturbing content. Harmful, hateful, violent, or sexually explicit 
‘search results are only appropriate if the person phrased their search in a way that makes it clear that they are looking for 
this type of content, and there is no other reasonable interpretation of the words used in their search, 





0.1 The Purpose of Search Quality Rating 


As a Search Quality Rater, you will help evaluate search engine quality around the world. Good search engines give 
results that are helpful for people in their specific language and locale. 


Your ratings will not direct affect how a particular webpage, website, or result appears in Google Search, nor wil they 
cause specific webpages, websites, or results to move up or down on the search results page. Instead, your ratings wil 
be used to measure how well search engine algorithms are performing for a broad range of searches. 


As part of your role in the search quality rating program, itis important that you are familiar with and comfortable using а 
search engine. We encourage you to be an expert in search! For example, experiment with using operators (e.9, quotes 
ога dash) in your searches or try using advanced search options. 

0.2 Raters Must Represent People in their Rating Locale 

itis very important for you to represent people in the locale you evaluate. You must be very familiar with the task language 


and location in order to represent the experience of people in your locale. If you do not have the knowledge to do this, 
Please inform your employer/company, 
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Unless your rating task indicates otherwise, your ratings should be based on the instructions and examples given in these 
guidelines. Ratings should not be based on your personal opinions, preferences, religious beliefs, or political views. 
Always use your best judgment and represent the cultural standards of your rating locale. 


03 Browser Requirements 


Check with your employericompany for browser requirements. You may use helpful browser add-ons or extensions, but 
please do not use add-ons or extensions that interfere with or alter the user experience of the page. 


04 Аа Blocking Extensions 


ро not use add-ons or extensions that block ads for Needs Met rating or Page Quality rating. These add-ons ог 
extensions may cause you to give incorrect ratings. Аз a rater, only use an ad blocking extension or add-on if specifically 
instructed to do so in the project-specific instructions. 


0.5 Internet Safety Information 


їп the course of your work, you will visit many different webpages. Some of them may harm your computer unless you 
are careful. Please do not download any executables, applications, or other potentially dangerous fles, or click on апу 
links that you are uncomfortable with 


Itis strongly recommended that you have antivirus and antispyware protection on your computer. This software 
must be updated frequently or your computer will not be protected. There are many free and for-purchase 
antivirus and antispyware products available on the web. 


‘See here for a Wikipedia page on antivirus software and here for a Wikipedia page on spyware. 


We suggest that you only open files with which you are comfortable. The file formats listed below are generally 
considered safe if antivirus software is in place. 


ва (text file) 
ppt or рріх (Microsoft PowerPoint) 
‘doc or docx (Microsoft Word) 

xls or xlsx (Microsoft Excel) 

pdt (PDF) fles. 


if you encounter a page with a warning message, such as "Warning-visiting this web site may harm your computer,” or if 
your antivirus software warns you about a page, you should not try to visit the page to assign a rating. 


You may also encounter pages that require RealPlayer or the Adobe Flash plugin. These are generally safe to download. 


0.6 Тһе Role of Examples in these Guidelines 


The examples throughout these guidelines are very important to ilustrate how the concepts defined in each section. 
should be applied їп rating tasks. Please view each example and keep the following in mind: 


= Webpages and websites change rapidly, so we use images or "snapshots" of webpages in most of our examples. 

‘© The information in the examples was accurate at the time it was added, but content and websites may change 
over time. 

‘© Some examples show pages on desktop and some show pages on mobile devices. 
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Part 1: Page Quality Rating Guideline 
1.0 Introduction to Page Quality Rating 


А Page Quaity (РО) rating task consists of a URL and a grid to record your observations, in order to guide your 
‘exploration of the landing page and the website associated with the URL. Ultimately, the goal of Page Quality rating is to 
evaluate how well the page achieves its purpose. Because diferent types of websites and webpages can have very 
different purposes, our expectations and standards for different types of pages are also different. 


Here's what you'll need to be a successful Page Quality rater 


‘© Your experience using the web as an ordinary user in your rating locale. 
‘In-depth knowledge of these guidelines. 
‘© And most importantly—practice doing РО rating tasks! 


2.0 Understanding Webpages and Websites 


РО rating requires an in-depth understanding of websites. We'll start with the basics. Along the way, well share 
important information about Page Quality rating, so please read through this section even if you are a website expert! 


24 — Important Definitions 
Here are some important definitions: 
A search engine is a tool to help people find or interact with content available on the Internet. 


In these guidelines, the word "user" refers to a person trying to find information or accomplish a task on the Internet. 
Keep in mind that users are people from many diferent backgrounds, whose experiences and needs may differ from your 
‘own: people of al ages, genders, races, religions, political affiliations, etc. 


A webpage is connected to the World Wide Web and can be viewed or "visited" using a web browser (e.g., Chrome), а 
browser on your phone, or a search app. In the 19805, webpage content was mostly text and links. Today, webpage 
Content includes many forms of media (such as images, videos, etc.) and functionality (such as online shopping features, 
email, calculator functionality, online games, etc). 


A URL is a character string that your web browser uses to “find” and display a webpage. Page Quality rating doesnt 
require you to have in-depth understanding of the structure of URLs, i.e., you don't need to know the difference between 
host, domain, eic. But if you are interested, see hore to read more. 


A website or site is a group of World Wide Web pages usually containing hyperlinks to each other and made available 
online by an individual, company, educational institution, government, or organization. Popular websites include 
Facebook, Wikipedia, Yahoo, YouTube, ete. 


Note: In these guidelines, we wil use the word "website" to refer to a collection of pages owned and controlled by a single 
entity (individual, business, etc). But we wil also use "website to refer to major "independent sections (or hosts) of 
‘some websites that were created to achieve separate purposes. For example, the Yahoo website is organized into 
different sections (or hosts), such as Yahoo Finance (finance. yahoo.com), Yahoo Май (mail.yahoo.com), Yahoo Sports 
(sports. yahoo.com), etc. Each of these has its own purpose. Its OK to refer to each of these sections as a website; for 
example, the Yahoo Finance website and the Yahoo Sports website. You may also refer to pages on Yahoo Finance or 
Yahoo Sports as belonging to the Yahoo website. 


А homepage of a website is the main page of the site. It is usually the first page that users see when the site loads. For 
‘example, http://www. apple.com is the homepage of the Apple site, http:Jiwww.yahoo.com is the homepage of the Yahoo 
‘company site, and hitp/finance.vahoo.com is the homepage of Yahoo Finance. You can usually find the homepage of a 
website by clicking on a "home" link or logo link on subpages of a website. 
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А subpage on a website is any page on the site other than the homepage. For example, hitp/iwwwapole.comlighone is 
а subpage on the Apple website, and hiips/finance, yahoo cominews is a subpage on the Yahoo Finance website. 


A wobmastor is the person who is responsible for maintaining a website. 


Important: You must be very comfortable exploring websites, both by clicking links and modifying URLs in the address 
bar of your web browser. Become a website detective and explorer! 


22 — Whatis the Purpose of a Webpage? 


The purpose of a page is the reason or reasons why the page was created. Every page on the Internet is created for a 
purpose, or for multiple purposes. Most pages are created to be helpful for people, thus having a beneficial purpose. 
Some pages are created merely to make money, with litle or no effort to help people. Some pages are even created to 
harm users. The first step in understanding a page is figuring out its purpose. 


Why is it important to determine the purpose of the page for РО rating? 


* The goal of PQ rating is to determine how well а page achieves its purpose. In order to assign a rating, you must 
understand the purpose of the page and sometimes the website. 

«By understanding the purpose of the page, you'll better understand what criteria are important to consider when 
evaluating that particular page. 

‘© Websites and pages should be created to help people. If that is not the case, a rating of Lowest may be 
warranted. More on this later. 


As long as the page is created to help people, we will not consider any particular page purpose or type to be higher 
Quality than another. For example, encyclopedia pages are not necessarily higher quality than humor pages. 


Important: There are highest quality and lowest quality webpages of all different types and purposes: shopping pages, 
news pages, forum pages, video pages, pages with error messages, PDFs, images, gossip pages, humor pages, 
homepages, and all other types of pages. The type of page does not determine the PQ rating—you have to understand 
the purpose of the page to determine the rating. 


Common helpful or beneficial page purposes include (but are not limited to 


‘© To share information about a topic. 
To share personal or social information 

To share pictures, videos, or other forms of media. 

To express an opinion or point of view. 

To entertain. 

То sell products or services. 

То allow users to post questions for other users to answer. 

To allow users to share fles or to download software. 


Here are a few examples where itis easy to understand the purpose of the page: 




















{Type of Page. Purpose of the Page 

[News website homepage [To inform users about recent or important events. 
|Shopping page. [To sell or give information about the product. 

Video page То share а cute video of a cat. 

(Currency converter page [To calculate equivalent amounts in different currencies. 
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Hero is an example (OmNomNomNom Раде) of а helpful page where the purpose of the page is not as obvious. At first 
glance, this page may seem pointless or strange. However, it is a page from a humorous site that encourages users to 
post photos with mouths drawn on them. The purpose of the page is humor or artistic expression. This page has а 
helpful or beneficial purpose. Even though the About page on this website is not very helpful, the website explains itself 
on its FAQ page. 


23 Your Money or Your Life (YMYL) Pages 


‘Some types of pages or topics could potentially impact a person's future happiness, health, financial stability, or safety. 
We cal such pages "Your Money or Your Life" pages, or YMYL. The following are examples of YMYL topics: 


* News and current events: news about important topics such as international events, business, politics, science, 
technology, ete. Keep in mind that not all news articles are necessarily considered YMYL (e.g. sports, 
entertainment, and everyday lifestyle topics are generally not YMYL). Please use your judgment and knowledge 
of your locale. 

* Civics, government, and law: information important to maintaining an informed citizenry, such as information 
about voting, government agencies, public institutions, social services, and legal issues (e.g. divorce, child 
custody, adoption, creating a wil, ec. 

* Finance: financial advice or information regarding investments, taxes, retirement planning, loans, banking, ог 
insurance, particularly webpages that allow people to make purchases or transfer money online. 

« Shopping: information about or services related to research or purchase of goods/services, particularly 
webpages that allow people to make purchases online. 

Health and safety: advice or information about medical issues, drugs, hospitals, emergency preparedness, how 
dangerous an activity is, etc. 

* Groups of people: information about or claims related to groups of people, including but not limited to those 
grouped on the basis of age, caste, disability, ethnicity, gender identity and expression, immigration status, 
nationality, race, religion, sex gender, sexual orientation, veteran status, victims of a major violent event and their 
kin, or any other characteristic that is associated with systemic discrimination or marginalization. 

* Other: there are many other topics related to big decisions or important aspects of people's lives which thus may 
be considered YMYL, such as fitness and nutrition, housing information, choosing a college, finding a job, etc. 
Please use your judgment. 








We have very high Page Quality rating standards for YMYL pages because low quality YMYL pages could potentially 
negatively impact a person's happiness, health, financial stability, or safety. 


24 Understanding Webpage Content 


Al of the content on a webpage can be classified as one of the following: Main Content (MC), Supplementary Content 
(SC), or Advertisements/Monetization (Ads). In order to understand the purpose of a webpage and do РО rating, you wil 
need to be able to distinguish among these different parts ofthe page. 


Webpage design can be complicated, so make sure to click around and explore the page. See what kind of content is 
behind the tabs and test out the interactive page features. Content behind the tabs may be considered part of the MC, 
SC, or Ads, depending on what the content is. 


24.1 Identifying the Main Content (MC) 


Main Content is any part of the page that directly helps the page achieve its purpose. Webmasters directly control ће MC 
of the page (except for user-generated content). MC can be text, images, videos, page features (e.g.. calculators, 
games), or it can be user-generated content such as Videos, reviews, articles, etc. that users have added or uploaded to 
the page. Note that tabs on some pages lead to even more information (e.g., customer reviews) and can sometimes be 
considered part of the MC of the page. 


The MC also includes the title at the top of the page (example). Descriptive MC titles allow users to make informed. 
decisions about what pages to visit. Helpful titles summarize the МС on the page. 
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‘Type of Page and Purpose мс Highlighted in Yellow 
News website homepage: the purpose is to inform users about recent or important events. [MC = News Homanage 
News article page: the purpose is to communicate information about an event or news tope. MME -News Arla 


‘Store product page: the purpose i to sell ог give information about he produc. 
‘© Content behind the Reviews, Shipping, and Safety Information tabs аге considered to be MC - Shopping Page 



































par of the MC. 
Video page: the purpose is to share а video about cats. |с - Video Pane 

(Currency converter page: the purpose is to calculate equivalent amounts in differant currencies. [MC - Currency Converter 

Blog post page: the purpose is to share music used on a TV show. IM - Bing Post Page 

‘Search engine homepage: the purpose is to allow users to enter a query and search the Internet. [MC Search Engine Homepage 
[Bank login page: the purpose is to allow users to log in ta bank online. [MC Bank Login Page 











2.4.2 Identifying the Supplementary Content (SC) 


‘Supplementary Content contributes to a good user experience on the page, but does not directly help the page achieve its 
purpose. SC is controlled by webmasters and is an important part of the user experience. One common type of SC is 
navigation links that allow users to visit other parts of the website. Note that in some cases, content behind tabs may be 
considered part of the SC of the page. 


Sometimes the easiest way to identify SC is to look for the parts of the page that are nat MC or Ads. 
































Туре of Page and Purpose. (SC Highlighted in Blue. 
News article pags: the purpose is to communicate information about an event or news topic- C= News Artide 
‘Sore product page: the purpose i to sell or ive information about the product. [SC Shonning Page 
|Video page: the purpose is to share а video about са. [SC Video Page 
[log post page: the purpose is to share music used on a TV show. ISC- Biog Post Page 





2.4.3 Identifying Advertisements/Monetization (Ads) 


Ads may contribute to a good user experience. Advertisements/Monetization (Ads) is content andlor inks that are 
displayed for the purpose of monetizing (making money from) the page. The presence or absence of Ads is not by itself a 
reason for a High or Low quality rating. Without advertising and monetization, some webpages could not exist because it 
costs money to maintain a website and create high quality content. 


There are several different ways to monetize a webpage, including advertisements and affiliate programs. See here for 
тоге information on website monetization. Note that monetization an mobile pages may be more subtle than 
monetization on desktop pages. 


The most common type of monetization is advertisements. Ads may be labeled as "ads," "sponsored links," "sponsored 
listings,” "sponsored results,” etc. Usually, you can click on the links or mouse over the content to determine whether they 
are Ads, as they often referto a URL outside of that website. Ads may change when you reload the page, and different 
users may see different Ads on the same page. 


Webmasters can choose to display Ads on thelr page (such as by joining an advertising network), but they may not 
always directly control the content of the Ads. However, we will consider a website responsible for the overall quality of 
the Ads displayed 


Important: For the purpose of this guideline, we will consider monetized links of any type to be "Ads.” See here for 
diferent types of website monetization 
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Туре of Page and Purpose. [Ads Highlighted in Red. 
News article page the purpose is to communicate information about an event or news tole. Ads - News Article 

|Video page: the purpose is to share а video about са. lads Video Page 

Blog post page: the purpose is to share music used on a TV show. lads Воз Post Page 
[Store product page: the purpose is to sell or give information about the produc. [No ads = Shopoing Pags. 











2.4.4 Summary of the 





rts of the Page 
Lets put it all together. 


‘© Main Content (MC) is any part of the page that directly helps the page achieve its purpose. MC is (or should bel) the 
reason the page exists. The quality of the MC plays a very large role in the Page Quality rating of a webpage. 

* Supplementary Content (SC) is also important. SC can help a page better achieve its purpose or it сап detract from 
the overall experience. 

‘© Many pages have advertisemonts/monetization (Ads). Without advertising and monetization, some webpages 
could not exist because it costs money to maintain a website and create high quality content. The presence or 
absence of Ads is not by itself a reason for a High or Low quality rating. 


Оп some pages, reviews may be considered MC, and on other pages they may be considered SC. Use your best. 
judgment and think about the purpose of the page. 


ро nat worry too much about identifying every litle part of the page. Think about which parts of the page are the MC. 
Next, look for the Ads. Anything left over can be considered SC. 












































Typa of Page and Purpose EU 
Few pope a pipes a e nen тсс 
Br иш рии: races sar rin йо а m Basan Sn гш 
io poe pons aparo a video a eais аныз аган 
arae anara pap Pe rr cian ИЕА a aw Bima билан So 
Bs ll paga he peas acd TV Suman анара 
Bb soe er lo eno rk SAE mta amicus 














2.5 Understanding the Website 


Pages often make more sense when viewed as рап of a website. Some of the criteria in Page Quality rating are based 
оп the website the page belongs to. 


In order to understand a website, look for information about the website on the website itself. Websites are usually very 
eager to tell you all about themselves! 


You must also look for reputation information about the website. What do outside, independent sources say about the 
website? When there is disagreement between what the website says about itself and what reputable independent 
‘sources say about the website, trust the independent sources. 

2.5/1 Finding the Homepage 


The homepage of a website usually contains ог has links to important information about the website. Webmasters usually 
make it easy to get to the homepage of the website from any page on the site. 
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Here's how to find the homepage of a website: 


* Examine the landing page of the URL in your РО rating task 
Find and click on the link labeled "home" or "main page." 

© Having trouble finding it? Try using "С.Е" ('command-F" on a Mac) to search the page for the text "home" ог 
“main.” You may also try clicking on the website logo, which is usually at the top of the page. 





‘Sometimes, you may be given a webpage or website that appears to have no navigation links, no homepage link, and по 
logo or other means to find the homepage. Even some High or Highest quality pages lack a way to navigate to the 
homepage. If you canit find a link to the homepage, modify the URL by removing everything to the right ог". сот" 
“ne,” “info,” etc. and refresh the page. 


org” 


Occasionally, your rating task vil include а URL for which there are two or more justifiable "homepage" candidates. For 
example, you may not be sure whether the homepage of the URL httos:/finance vahoo.com/calendar is 
http:lfînance yahoo.com or http:/iwww.yahoo.com. 


Important: When you have more than one homepage “candidate,” please use whichever one offers the most information. 
about the specific webpage in the rating task. Use your judgment. The goal is to understand the webpage and the 
website(s) it is associated with, not find the one unique, correct homepage. 


In the folowing examples, we have included the URL of the page to be evaluated in the rating task, as well as the URL of 
Из associated homepage. We have also included an image that shows where to cick on the landing page to navigate to 
the homepage. In the image, you will see a red box around the link or logo you would click to navigate to the homepage. 





Image that shows where to click to gat to the 


URL ofthe Task Page | Homepage of the Website eas tt 








WILLIAMS 
istam sonomae 


235 | nip wiliams-sonoma com SONOMA 
шасе ша! 
This "WILLIAMS-SONOMA logo shown in the 


upper center of the page is clickable and takes 
users to the homepage of the website 








toms harvard edu 


Tn this case, we wil consider to Harvard Medical HARVARD 
оо page at hb. hms haran ad tobe the 
‘School page at 1o bo the MEDICAL SCHOOL 





trs harvard edulabout: | homepage, rather than Шр аш hanan edu 
табаса figures (which is the homepage of Harvard Universi). 
‘licking the logo at the top of “This "Harvard Medical Schoo! logo in the upper 
himla harvard ашаа шоа ша ей part of he page is clickable and takes users 
fakes users to hip hms harvard.sdu, nolo ^ о the homepage of the Harvard Medical School 
шоа агаа adu website 
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2.5.2 Finding Who is Responsible for the Website and Who Created the Content on tho Page 
Every page belongs to a website, and it should be clear: 


= Who (what individual, company, business, foundation, etc.) is responsible for the website. 
‘© ^ Who (what individual, company, business, foundation, etc.) created the content on the page you are evaluating. 


Websites are usually very clear about who created the content on the page. There are many reasons for this: 


Commercial websites may have copyrighted material they want to protect. 
Businesses want users to know who they are. 

Artists, authors, musicians, and other original content creators usually want to be known and appreciated. 
Foundations often want support and even volunteers. 

High quality stores want users to feel comfortable buying online. 

Websites want users to be able to distinguish between content created by themselves versus content that was 
added by other users. 


Most websites have “contact us" or “about us" or “about” pages that provide information about who owns the site. Many 
companies have an entire website or blog devoted to who they are and what they are doing, what jobs are available, ete. 
Google and Marriott are both examples of this, and there are many others: 


Google Oficial Blog 
Marriott Blog 
Southwest Airines Biog 
Мет» Tech Blog 


Often a business or organization is responsible for the content of a website, not an individual person. The IBM 
Corporation is responsible for the content on ibm.com. The Cleveland Clinic is responsible for the content on 
clevelandclinic.org. An individual is not responsible for the content on these websites, even though many individuals 
Contributed to creating and maintaining the content. In these cases, we will view the business or organization as 
responsible for the content on every single page, as well as maintenance of the website. 


Оп some websites, users create the MC of many pages, while the business or organization itself maintains the website. 
The company Facebook is responsible for the Facebook website, but individuals create the content on their personal 
Facebook pages. The company Wikipedia is responsible for the Wikipedia website, but individuals create article content. 
Other websites with user-generated content include YouTube, Twitter, other social networking websites, other article 
publishing websites, Q&A websites, forums, etc. For these websites, you must look at each page to determine the 
author(s) ог creator(s) of the content on that page. 


Finally, there are some websites that show licensed or syndicated content. This means that the website has paid money 
ог has some business relationship with the creator of the content. In these cases, we wil consider the website to carry 
responsibility for the quality of licensed or syndicated content, even if it wasn't created by the website itself. 


2.8.3 Finding About Us, Contact Information, and Customer Service Information. 


Many websites are interested in communicating with their users. There are many reasons that users might have for 
contacting a website, from reporting problems such as broken pages, to asking for content removal. Many websites offer 
‘multiple ways for users to contact the website: email addresses, phone numbers, physical addresses, web contact forms, 
ete. Sometimes, this contact information is even organized by department and provides the names of individuals to 
contact. 
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‘The types and amount of contact information needed depend on the type of website. Contact information and customer 
service information are extremely important for websites that handle money, such as stores, banks, credit card 
companies, elc. Users need a way to ask questions or get help when a problem occurs. 


For shopping websites, we'll ask you to do some special checks. Look for contact information—including the store's 
policies on payment, exchanges, and retums. Sometimes this information is listed under “customer service." 


‘Some kinds of websites need fewer details and a smaller amount of contact information for their purpose. For example, 
humor websites may not need the level of detailed contact information we would expect from online banking websites. 


Occasionally, you may encounter a website with а legitimate reason for anonymity. For example, personal websites may 
not include personal contact information such as an individual's home address or phone number. Similarly, websites with 
user-generated content may allow the author to identify him/herself with an alias or username only. 


To find contact or customer service information for a website, start with the homepage. Look for а "contact us" or 
"customer service" link. Explore the website if you cannot find a "contact us" page. Sometimes you wil find the contact 
information on a "corporate site" link or even on the company's Facebook page. Be a detective! 


Note that different locales may have their own specific standards and requirements for what information should be 
available on the website. 


2.6 Reputation of the Website and Creator of the Main Content 


‘An important part of PQ rating is understanding the reputation of the website. If the creator of the MC is diferent from the 
creator of the website, i's important to understand the reputation of the creator as well 


‘A website's reputation is based on the experience of real users, as well as the opinion of people who are experts in the 
topic of the website. Keep in mind that websites often represent real companies, organizations, and other entities. 
Therefore, reputation research applies to both the website and the actual company, organization, or entity that the website 
is representing 


Knowing more about the reputation of a website and content creator can also help you understand what a website is best 
known for, and as a result how wel it accomplishes its purpose. For example, newspapers may be known for high quality, 
independent investigative reporting while satire websites may be known for their humor. 


Many websites are eager to tell users how great they are. Some webmasters have read these rating guidelines and write 
"reviews? on various review websites. But for PQ rating, you should aim to find independent sources of reputation. 
information about the website and creator of the MC rather than relying solely on what the website itself or content creator 
has to зау. 
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Your job is to evaluate the reputation of the website and creator of the MC. Please research the website and content 
creator to find out what other people and experts have to say. 


2.6.1 Research on the Reputation of the Website and Creator of the Main Content 





Use reputation research to find out what real users, as well as experts, think about a website. Look for reviews, 
references, recommendations by experts, news articles, and other credible information created/wrtten by individuals 
about the website 


User reviews are often helpful sources of reputation for websites that offer products or services. You may consider a large 
number of detailed, trustworthy positive user reviews as evidence of positive reputation. 


For other kinds of websites, sources of reputation information will vary according to the topic or type of 
'companylorganization/entity that the website represents. For example, you might find that a newspaper (with an 
associated website) has won journalist awards. Prestigious awards or a history of high quality original reporting are 
strong evidence of positive reputation 


For individual authors and content creators, biographical information articles can be a good source of reputation. 
information. 


For YMYL informational topics, the reputation of a website or content creator should be judged by what experts in the field 
have to say. Recommendations from expert sources, such as professional societies, are strong evidence of a very 
positive reputation 


Carefully consider the purpose of the page, whether or not the topic is YMYL, and the kind of reputation information that 
would be most applicable. For example, customer ratings and reviews may be helpful for reputation research of online 
stores, but much less so for medical information websites. 


Reputation research is necessary for all websites and content creators you encounter, to the extent that an established 
reputation can be found. Do not just assume websites you personally use have a good reputation. Please do research! 
You might be surprised at what you find 


2.6.2 Sources of Reputation Information 
Look for information written by a person, not statistics or other machine-compiled information. News articles, Wikipedia 
articles, blog posts, magazine articles, forum discussions, and ratings from independent organizations can ай be sources 
of reputation information. Look for independent, credible sources of information. 

‘Sometimes, you will ind information about a website that is not related to its reputation. For example, pages like Alexa 
have information about Internet traffic to the website, but do not provide evidence of positive or negative reputation. You 
сап ignore this information since its nat helpful for Page Quality rating. 


2.6.3 Customer Reviews of Stores/Busin 





Customer reviews can be helpful for assessing the reputation of a store or business. However, you should interpret these 
reviews with care, particulary if there are only a few. Be skeptical of both positive and negative user reviews. Anyone can 
write them, including the creator of the website or someone the store or business hires for this purpose. 


When interpreting customer reviews, try to find as many as possible. Any store or business can get a few negative 
reviews. This is completely normal and expected. Large stores and companies have thousands of reviews and most 
receive some negative ones. 


Itis also important to read the reviews because the content of the reviews matter, not just the number. Credible 
convincing reports of fraud and financial wrongdoing is evidence of extremely negative reputation. A single encounter. 
with a rude clerk or the delayed receipt ofa single package should not be considered negative reputation information. 
Please use your judgment. 
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2.6.4 How to Search for Reputation Information 


Here is how to research the reputation of the website: 


1 


Identify the "homepage" of the website. For example, for the IBM website, от сот is the homepage. You тау 
need to identify the creator of the content, if it is different from that of the overall website. 


Using ibm.com as an example, try one or more of the following searches on Google 


[ibm -sitebm.com]: A search for IBM that excludes pages on ibm.com. 
Tibm.comr -site ibm com] A search for "ibm.com" that excludes pages оп ibm.com. 

[ibm reviews -steibm.com A search for reviews of IBM that excludes pages on ibm.com. 

[ibm.conr reviews -siteibm.com] A search for reviews of "ibm.com" that excludes pages on ibm.com. 
For content creators, try searching for their name or alias. 


Note: When searching for reputation information, try to find sources that were not written or created by the 
website, the company itself, or the individual. For example, IBM might have official Facebook or Twitter pages 
that it closely maintains, which would not be considered independent sources of reputation information about the 
company. See here for a Wikipedia article on identifying and using independent sources. 


Look for articles, references, recommendations by experts, and other credible information written by people about 
the website or the content creator. For businesses, look Гог reputation information and reviews from a variety of 
sources. For content creators, look for biographical data and other sources that are nat written by the content. 
creator themselves. 


High quality news articles and informational articles may be good sources of information for both 
‘companiasforganizationsiantities and content creators. Search for such articles. For example, try [bm 

site:en wikipedia org] oc ibm. com site:en wikipedia org]. News articles and informational articles can help you 
learn about a company and may include information specific to reputation, such as awards and other forms of 
recognition, or also controversies and issues. Note that some informational articles include a message warning 
users that there are disagreements on some of the content, or that the content may be outdated. This may be an 
indication that additional research is necessary. 


Make sure the information you find is appropriate for judging the reputation of the website. For example, 
reputation information for YMYL websites or YMYL content creators should come from sources that have 
expertise in the associated YMYL topic. 


Here are some examples of reputation information 





Wobsite Reputation Information About the Site Description 








Positive reputation information: Users in the U.S. 
сап obtain free credit reports on this website by 
providing their Social Security Number. Nate that 


Search results tor Jannualerediireport com ке Wikipedia arici telis us that 
le annvaleredtrepor сот "AnnalCredtReport.com isthe only federally 


‘mandated and authorized source for obtaining а 


шшш cam wikipedia aricle about annualcrediireporkeom | free credit report” 


Вараш article about annualeradiirenortsom | Note: Almost every website will have complaints 
about customer service, во itis important to lok at 
Various sources and reviews in your reputation 
search, 














Copyright 2021 т 








Reputation Information About the Site 


Description 





|cievelandeleic ora 


Search results tor [clevelandzlic org 


Wikipedia arici about clevtandeleic ora 


Positive reputation information: According to 
Wikipedia, the Cleveland Січ "s currently 
regarded as one of the top 4 hospitals in the United 
States" which can be confirmed by reputable news 
апаз cited in fn references section. Users can 
rust medical information on this website. 








Positive reputation information: Notice the 
highiignted section in the Wikipedia article about 
The Christian Science Manitor newspaper, which 
tells us that the newspaper has won several 
prestigious awards. From this information, it can be 
{fered that the esmonitor com website has a 


positive reputation. 





Search results tor [kemeLong -stexernelar 
Wikipedia article about кеше org 


Positive reputation information: According to the, 
Wikipedia article, “Кете! огу is а main repository of 
Source cada ar ina Linux kernel, the base ofthe 
Popular Linux operating system. It makes al 
Versione of he source code available lo all users. I 
aiso hosts various other projects, ike Google 
Android. The main purpose of the site is to host a 
repository for Linux kemat developers and 
maintainers ef Linux distributions. 





Ste satina children's 


Search to find reputation information 
Search to ind reviews 





Extremely negative reputation information: This 
business has a very low rating on a business review 





Negative n 2051 sile. There is a news article about financial fraud 
Hm Negative review on a business review nage 2 | There are many reviews on websites describing 
users sending money and nat receiving anything. 
Negative news article 
Search ia tina reputation information 
Negative review on a business review page Extremely negative/malicious reputation 
ао aaan information: This website engaged in criminal 
imamdlosvenear Wikipedia acide 
behavior such as physically threatening users. 
Aricie about businass criminal behavior 





hespratzed veteran 
ашу 





Negative review 1 
Negative гашен 2 
Negative review 3 
Negative review 4 








Extremely negative reputation information: 
‘There are many detailed negative articles on news 
tes and chart watchdog sites about thie 
organization describing fraud and financial 
‘mishandling 








2.6.5 What to Do When You Find No Reputation Information 


You should expect to find reputation information for large businesses and websites of large organizations, as well as 
well-known content creators. 


Frequently, you will find litle or no information about the reputation of a website for a small organization. This is not 
indicative of positive or negative reputation. Many small, local businesses or community organizations have a small web 
Presence" and rely on word of mouth, nat online reviews. For these smaller businesses and organizations, lack of 


reputation should not be considered an indication of low page quality. 
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3.0 Overall Page Quality Rating 
Ata high level, here are the steps of Page Quality rating: 


1. Assess the true purpose of the page. 
Assess the potential of the page to be harmful, untrustworthy, or spammy as defined in Section 7.0 of these. 
guidelines. 

3. Otherwise, the РО rating is based on how well the page achieves its purpose using the criteria outlined in the 
following sections on Lowest, Low, Medium, High, and Highest quality pages. 


Оп Page Quality rating tasks, you will use the Page Quality siding scale (sider) to assign the overall PQ rating. The 
slider looks like this: 


0 П 1 1 ] 


You may also use the in-between ratings of Lowest*, Lows, Mediums, and High. Please interpret the "+" as *+ %4," 
meaning that the Loweste rating is halfway between Lowest and Low, Lowe is halfway between Low and Medium, etc. 


In the following sections, you will learn about characteristics of Lowest, Low, Medium, High, and Highest quality pages. 
зл Раде Quality Rating: Most Important Factors 


Here are the most important factors to consider when selecting an overall Page Quality rating: 


‘© The Purpose of the Page 

* Expertise, Authoritativeness, Trustworthiness: This is an important quality characteristic. Use your research 
оп the additional factors below to inform your rating 

= Main Content Quality and Amount: The rating should be based on the landing page of the task URL. 

‘© Website Informationlinformation about who is responsible for the MC: Find information about the website as 
жей as the creator of the MC. 

< Website Reputation/reputation about who is responsible for the МС: Links to help with reputation research 
will be provided. 


Note: Some tasks may ask you to view the page on your phone, but to do research (e.g., finding website information and 
reputation) on your desktop. Other tasks may ask you to do everything on desktop. Please follow instructions in the task. 


3.2 Expertise, Authoritativeness, and Trustworthiness (E-A-T) 
The amount of expertise, authoritaiveness, and trustworthiness (E-A-T) is very important. Please consider: 
* The expertise of the creator of the MC. 


The authoritativeness of the creator of the MC, the MC itself, and the website. 
‘© The trustworthiness of the creator of the MC, the MC itself, and the website. 
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Keep in mind that there are high Е-А-Т pages and websites of ай types, even gossip websites, fashion websites, humor 
Websites, forum and ОВА pages, etc. In fact, some types of information are found almost exclusively on forums and 
discussions, where a community of experts can provide valuable perspectives on specific topics. 


‘© High E-A-T medical advice should be written or produced by people or organizations with appropriate medical 
expertise or accreditation. High E-A-T medical advice or information should be written or produced in а 
professional style and should be edited, reviewed, and updated on a regular basis. 

* High E-A-T news articles should be produced with journalistic professionalism—they should contain factually 
accurate content presented in a way that helps users achieve a better understanding of events. High E-A-T news 
Sources typically have published established editorial policies and robust review processes (example 1, example 
3. 

«^ High E-A-T information pages on scientific topics should be produced by people or organizations with appropriate 
scientific expertise and represent well-established scientific consensus on issues where such consensus exists. 

‘© High E-A-T financial advice, legal advice, tax advice, ete., should come from trustworthy sources and be 
maintained and updated regularly. 

‘© High E-A-T advice pages on topics such as home remodeling (which can cost thousands of dollars and impact 
your living situation) ог advice оп parenting issues (which can impact the future happiness of a family) should also 
come from "expert or experienced sources that users can trust. 

‘© High E-A-T pages on hobbies, such as photography or learning to play a guitar, also require expertise. 


‘Some topics require less formal expertise. Many people write extremely detailed, helpful reviews of products ог 
restaurants. Many people share tips and Ife experiences on forums, blogs, etc. These ordinary people may bo. 
considered experts in topics where they have life experience. If it seems as if the person creating the content has the. 
type and amount of life experience to make him or her an “expert” on the topic, we will value this ‘everyday expertise” and 
not penalize the person/webpagelwebsite for not having “formal” education or training in the field. 








ifs even possible to have everyday expertise in YMYL topics. For example, there are forums and support pages for 
people with specific diseases. Sharing personal experience is a form of everyday expertise. Consider this example. 
Here, forum participants are telling how long their loved ones lived with liver cancer. This is an example of sharing 
personal experiences (in which they are experts), not medical advice. Specific medical information and advice (rather 
than descriptions cf life experiences) should come from doctors or other health professionals. 


Think about the topic of the page. What kind of expertise is required for the page to achieve its purpose неї? The 
standard for expertise depends on the topic of the page. 
4.0 High Quality Pages 


In this section, we will describe characteristics of High quality pages and give many examples. Examples help calibrate 
your ratings, so please review each опе. 


44 — Characteris 





of 





п Quality Pages 


High quality pages exist for almost any beneficial purpose, from giving information to making people laugh to expressing 
oneself artistically to purchasing products or services online. 


What makes a High quality page? A High quality page should have a beneficial purpose and achieve that purpose well 
In addition, High quality pages have the following characteristics: 


» High level of Expertise, Authoritatveness, and Trustworthinoss (Е-А-Т). 

‘© А satisfying amount of high quality MC, including a descriptive or helpful tile. 

+ Satisfying website information andlor information about who is responsible for the website. 1f the page is primarily 
for shopping or includes financial transactions, then it should have satisfying customer service information. 

‘© Positive website reputation for a website that is responsible for the MC on the page. Positive reputation of the 
creator of the MC, if different from that of the website. 
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42 ASatisfying Amount of 





h Quality Main Content 


The quality of the MC is one of the most important criteria in Page Quality rating, and informs the E-A-T of the page. For 
all types of webpages, creating high quality MC takes a significant amount of at least one of the following: time, effort, 
expertise, and talent/skil. For news articles and information pages, high quality MC must be factually accurate for the 
topic and must be supported by expert consensus where such consensus exists. 


For each page you evaluate, spend a few minutes examining the MC before drawing a conclusion about it. Read the 
article, watch the video, examine the pictures, use the calculator, play the online game, etc. Remember that MC also 
Includes page features and functionality, so test the page out. For example, if the page is a product page on a store 
website, put at least опе product in the cart to make sure the shopping cart is functioning. If the page is an online game, 
spend a few minutes playing it. 


‘The purpose of the page will help you determine what high quality content means for that page. For example, High quality 
information pages should be factually accurate, clearly written, and comprehensive. High quality shopping content should 
allow users to find the products they want and to purchase the products easily. High quality humor ог satire should be 
entertaining, while factual accuracy is not a requirement as long as the page would be understood as satire by users. 


The amount of content necessary for the page to be satisfying depends on the topic and purpose of the page. A High 
quality page on a broad topic with a lot of available information will have more content than a High quality page on а 
narrower topic. Here are some examples of pages with a satisfying amount of high quality MC. 





Webpage Discussion 





Siberian Husky | The Siberian Husky (s breed of dog) is a narrow topic. Although this encyclopedia landing page has lass MC. 
Information Page | һап some encyclopedia pages on broader topics, it has a satisfying amount of clearly writen, high quality МС. 





This shopping page on a reputable shopping website has а satisfying amount of high quality MC. The page 
provides tha manufacturer s produc! specs, as well as original product information, over 90 user reviews, 
Shipping and returns information, multiple images ofthe product, lc. Note: Some of the МС в behind links on 
the page (item delals "tem specifications," "guest reviews,” oti). Even though you have о click these inks 
чо see the content, i stil considered MC. 


Kitchen Stand Mixer. 
Shopping Page 








Tis movie review writen by a movie critic has a satisfying amount of high quality МС. Timo, effort, and 


love Review Page 
Mowe Review Pane nti want inte wring this movie review. 














4.3 Clear and Satisfying Website Information: Who is Responsible and Customer Service 


Understanding who is responsible for a website is a critical part of assessing E-A-T for most types of websites. High 
quality pages should have clear information about the website so that users feel comfortable trusting the site. 


The amount of information needed for E-A-T assessment depends on the type of website. For example, YMYL websites 
demand a high degree of trust, so they generally need satisfying information about who is responsible for the content of 
the site. In addition, High quality stores and financial transaction websites also need clear and satisfying customer service 
Information to help users resolve issues. 


Other websites that are not YMYL websites may need less website information, depending on the purpose of the website. 
For example, an email address may be sufficient for some non-YMYL websites. 


44 — Positive Reputation 
Reputation is an important criteria when using the High rating, and informs the E-A-T of the page. While a page can merit 
the High rating with no reputation, the High rating cannot be used for any website that has a convincing negative 


reputation. Remember that when doing research, make sure to consider the reasons behind a negative rating and not 
just the rating itself 
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A High Level of Expertise/Authoritativeness/Trustworthiness (E-A-T) 


High quality pages and websites need enough expertise to be authoritative and trustworthy on their topic. Remember 
that there are “expert” websites of all types, even gossip websites, fashion websites, humor websites, forum and Q&A 
pages, etc. In fact, some types of information are found almost exclusively on forums and discussions, where а 
‘community of experts can provide valuable perspectives on specific topics. 


Think about the topic of the page. What kind of expertise is required for the page to achieve its purpose well? The 
standard for expertise depends on the topic of the page. For example, high quality news articles and information pages 
оп scientific topics should represent established scientific consensus where such consensus exists. 








46 Examples of High Quality Pages 
WebpageflypeofContent [High Quality Characteristics Rating and Explanation. 
Hoh: Nens 1 um з ен 


= Авайыўпа amount of high quality MC 
* ^ Positive reputation (website) 


‘This в the homepage of a newspaper that has 
Won several prestigious awards. This 
newspaper has а positive reputation for its 
objective reporting. 





High: News 2 


Arca on a newspaper website 
wi) 


e A satisfying amount of high quality MC 
* ^ Positive reputation (website) 
= High EAT of the publisher andlor author. 


= 


“This в an article on a newspaper website that 
has won severa prestigious awards and has а 
postive reputation for ts investigative 
Joumalism. There is a satisfying amount of 
high qualty MC. While there are Ads on the 
Page, е MC is easy to find and the Ads are 
early labeled as Ads. 





Hin: News 3 


Opinion article on a newspaper 
‘website 


© Asatistying amount of high quality MC 
Positive reputation (website) 
High E-A-T of the publisher andlor author. 


سسا 


— | 


This is an opinion piece writen by tno Editorial 
Board of tha largest newspaper in the U.S. 

state of Minnesota. The page is clearly labelled 
as ап opinion piece, welcoming a new football 
coach to the University of Minnesota. 





High: Government agency 


US Naval Observatory Master 
Clock page 








* High EAT forthe purpose of the page 





bobo d 


iem o Mem Highest 


The purpose of this page is ta display the 
official US Naval Observatory Master Clock 
time in T diferent ime zones. The page 
displays the cock information in a clear, 
easy-to-read format. The Naval Observatory is 
bight trustworthy and authoritative for this type 
‘of infomation. 
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Wobpage/Type of Content 


High Quality Characteristics. 


PQ Rating and Explanation 





High: Humor 


Алде on a humor website 


© High quality humorous МС 
Positive roputaton (website) 
+ Exporiso аз a farcical humor website 


ا | 


This website is well-known for its humorous, 
satirical articles. This в a cute example ofa 
Satisfying and funny artc. 





High: Small busines 


Local fish & chips restaurant 


^ High EAT for the purpose of tho page 


Gramas smana. 


This is an “about us" page on a restaurant 
website. This page provides information on 
when the restaurant opened and what visitors 
сап expect. Other pages on the website 
provide information about to restaurant 
Including the address, menu, ther contact 
information. etc. This website is highly 
aulhoriaiva because й а about sel. 





High: Small business 2 


Local preservation center 


High E-A-T forthe purpose of the page 
© A satisiying amount of high quality MC 


кее 


= 


This is the News and Updates section of a 
аса preservation center seling poultry, 
‘vegetables, and тоге. 


Note: This example was added in 2014 so the 
"News and Updates" wore timely. 





High: Bing nost 


Parenting article about strollers 


‘© A satisfying amount of high quality MC 
Very positive reputation (website) 

2 Specifically, high E-A-T and postive 
reputation for this specific blog and author 


=f 


This в а blog post on a newspaper that has 
won several prestigious awards. The author of 
iis blog post has become known as an expert 
оп parenting issues. Sho s a regular 
ontributor to ni and other media websites. 





High: Shopping 1 


Backpack shopping page on a 
‘popular store website (YMYL) 


e А satisfying amount of high quality MC 
e Positive reputation (website) 


fm 
Lort Medium Highest 
The purpose of this page is to allow users to 
buy а school backpack The page provides а 
Jot of ferent backpack options, and some of 
them have user reviews. 


This is а wellknown, reputable merchant, wilh 
detailed Customer Service information on Ine. 
ES 








High: Shopping 2 


Bathroom décor page on а store 
website (YMYL) 





* High E-A-T fr tho purpose of the page 
+ Positive reputation (website) 





ir 


-ıı 


This company sells its own line af high end, 
fashionable baby and children's furniture and 
accessories. I has postive reputation as well 
as expertise in these specie types af goods 
Many products sold on the site are unique to 
this company. 
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WabpagerType of Content 


High Quality Characteristics 


PQ Rating and Explanation 





High: Snooping 3 


‘TomTom GPS page on an 
lecronics website (YMYL) 





‘© А satisfying or comprehensive amount of 
very high quality MC 

= High EAT for the purpose of the page 

= Positive reputation (website) 


There is a very large quantity of MC on this 
page. Note thatthe tabs onthe page lead to 
‘even more information, incuding many 
‘customer reviews. The tabs shouid be 


considered part of the MC. 





High: Video 1 


Saturday Night Live video on a 
video website. 


‘© А satisfying or comprehensive amount of 
very high quality MC. 
= High EAT for the purpose of the page 


аа 


-1 


The MC of this video page is an episode of 
Saturday Night Live, а very popular television 
show. 





High: Video 2 


"An Engineer's Guide to Cats" 
video 


‘© А satisfying amount of high quality MC 
* High EAT (everyday expertise) on the 
topic of eat ownership 


[in 
leen Medium Highest 
“This в a humorous, high quality, wel produced 
video wilh more than 6.5 milion мен (and 
counting) ülustrating the proper care and 
practical benefits of cats. The two engineers in 
the video have everyday expertise on cat 
ownership. Fora humorous video ike this one, 
‘expertise does not have о be in the farm of a. 
Trained veterinarian. 











f manatee) 
High: Game = A satisfying or comprehensive amauntof | Lene Medium Highest 
very high quality MC 
Onine game © Positive reputation (website) as an Highly engaging game with тиде levels that 
‘educational too for kids could entertain а chid for hours. This website 
has а positive reputation for educating children 
арол animale, 
dem p pnma 
leen Medium Highest 
High: Q&A 


How lang da cancer patients ive 
vw) 





‘© А satisfying amount of high quality MC in 
{he form of personal narratives 





“The question on tha page asks how long 
people ve with cancer. There are many 
responses describing how long a loved one 
lived after diagnosis. There is very litle 
medical advice and the focus of the page is 
Sharing personal experiance. Many responses 
ara heartfelt and well writen, 
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WobpagerType of Content 


High Quality Characteristics 


PQ Rating and Explanation 





High: Encyclopedia 


Encyclopedia article about the 
American Civi War 


* A satisfying or comprehensive amount of. 
high quality MC 

= Positive reputation (website) 

$ High EAST for the article 


pepet 


= 


This в a detailed article about the American 
Civi War. Tha claions support the E-A-T of 
this rice 


Note: Although much of the contant is visible on 
е mobile page, we consider the content under 
the headings (which you need to expand) to be 
part of the MC. (Ta see the screenshot, you wil 
eed to zoom in to the image.) 





High: Forum 1 


Authenticating a luxury designer 
purse 


High EAT forthe purpose of the page 
© Asatsiying amount of high quality MC 


MÀ 


-i 


This forum is well-known for discussions on 
luxury designer purses. On this particular 
forum page, members are consulting forum 
experts who have expertise authenticating bags 
from this brand. These experts can tall if a 
particular bag is authentic or fake. While there 
езп ad at е top and a few ads within the 
forum message, it does not distract from the 
МС. which is easy to find. 


(то seo the screenshot, you wil need to zoom 
into the image.) 








Highs Forum 2 


Landscaping an aquarium 





High E-A-T for the purpose of the page 
A satisfying amount of high quality МС 





—є—— 


leen Mason о шш 


“This discussion focuses on the landscaping for 
а particular paludarum (an aquarium with 


terrestrial and aquatic elements), There is a 
{ot of discussion and interaction between forum 
members about the types of materials and 
Species used in tha aquarium. The posts show 
experisa in а niche opis aquarium 
landscaping 
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50 Highest Quality Pages 


Highest quality pages are created to serve a beneficial purpose and achieve their purpose very well. The distinction 
between High and Highest is based on the quality and quantity of MC, as well as the level of reputation and E-A-T. 


What makes а page Highest quality? In addition to the attributes of a High quality page, a Highest quality page must 
have atleast one of the following characteristics 


© Very high level of Expertise, Authoritativeness, and Trustworthiness (E-A-T). 

* A very satisfying amount of high or highest quality MC. 

‘© Very positive website reputation for a website that is responsible for the МС on the page. Very postive reputation 
of the creator of the MC, if different from that of the website. 


5A Very 





jh Quality MC 


We will consider the MC of the page to be very high or highest quality when itis created with a high degree of time and 
effort, and in particular, expertise, talent, and skiltis may provide evidence for the E-A-T of the page. Very high quality 
МС may be created by experts, hobbyists, or even people with everyday expertise. Our standards depend on tho. 
Purpose of the page and the type of content. For YMYL topics, there is а high standard for accuracy and well-established 
‘medical/scientic/historical consensus where such consensus exists. 





A factor that often distinguishes very high quality MC is the creation of unique and original content for the specific website. 
While what constitutes original content may be very different depending on the type of website, here are some examples: 


* For news: very high quality MC is original reporting that provides information that would not otherwise have been 
known had the article not revealed it. Original, in-depth, and investigative reporting requires a high degree of skill, 
time, and effort. Often very high quality news content will include a description of primary sources and other 
original reporting referenced during the content creation process. Very high quality news content must be 
accurate and should meet professional journalistic standards. 

« For artistic content (videos, images, photography, writing, elc): very high quality MC is unique and original 
content created by highly skilled and talented artists or content creators. Such artistic content requires a high 
degree of skilltalent, time, and effort. If the artistic content is related to a YMYL topic (е... artistic content with 
the purpose of informing or swaying opinion about YMYL topics), YMYL standards should apply. 

For informational content: very high quality MC is original, accurate, comprehensive, clearly communicated, 
professionally presented, and should reflect expert consensus as appropriate. Expectations for different types of 
Information may vary. For example, scientific papers have a diferent set of standards than information about a 
hobby such as stamp collecting. However, ай types of very high quality informational content share common 
attributes of accuracy, comprehensiveness, and clear communication, in addition to meeting standards 
appropriate to the topic or field. 


The Highest rating may be justified for pages with a satisfying or comprehensive amount of very high quality MC. 
52 — Very Positive Reputation 


Extensive reputation research is important when giving Highest ratings. Very positive reputation is often based on 
prestigious awards or recommendations from known experts or professional societies on the topic of the page. Wikipedia 
and other informational sources can be a good starting point for reputation research. 


For YMYL topics especially, careful checks for reputation are required. YMYL reputation should be based on evidence 
from experts, professional societies, awards, etc. For shopping pages, experts could include people who have used the 
store's website to make purchases; whereas for medical advice pages, experts should be people or organizations with 

appropriate medical expertise or accreditation. Please review Section 2 3 for a summary of types of YMYL pages/topics. 


For some topics, such as humor or recipes, less formal expertise is OK. For these topics, popularity user engagement, 
and user reviews can be considered evidence of reputation, For topics that need less formal expertise, websites can be 
considered to have a positive reputation if they are highly popular and well-loved for their topic or content type, and are 
focused on helping users. 
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Very High Level of E-A-T 


Highest quality pages and websites have a very high level of expertise or are highly authoritative or highly trustworthy. 


Formal expertise is important for YMYL topics such as medical, financial, or legal advice. Expertise may be less formal 
for topics such as recipes or humor. An expert page on cooking may be a page on a professional chefs website, or it may 
ве a video from an expert content creator who uploads very high quality cooking videos on YouTube and is one of the 
most well-known and popular content creators for recipes in their rea of expertise. Please value life experience and 
"everyday expertise" as appropriate. For some topics, the most expert sources of information are ordinary people sharing 
their life experiences on personal blogs, videos, forums, reviews, discussions, etc. 


Think about what expertise, authoritativeness, and trustworthiness mean for the topic of the page. Who are the experts? 
What makes a source trustworthy for the topic? What makes a website highly authoritative for the topic? Standards for 
very high E-A-T wil differ depending on the topic of the page. YMYL topics wil require higher standards. 








54 Examples of Highest Quality Pages 
Wobpage/Type of Content — | Highest Quality Characteristics PQ Rating and Explanation 
Jas paso pos eng 
+ Varyhigh level of EA-T for he purpose | Lanet Wasim Maren 
of the page, including ofthe publisher 


Arica on a newspaper website 
wi) 


andlor authors for a news article 

= Азайыўпа or comprehensive amount of. 
very high quality MC for the purpose of. 
the page 

‘© Very positive reputation (website) 


This в a feature article from a newspaper that 
has won numerous awards. 


The page has a comprehensive amount of very 
highquality MC. including in-depth reporting by 
two authors with extensive knowledge and 
experience in investigative joumalism. 





Highest: News 2 


Алде on a newspaper website 


‘Very high level of E-A-T for the purpose of 
же page, including ofthe publisher andlor 
authors for a news arice 

‘© Азайыўпа or comprehensive amount of 
very high quality MC for the purpose of 
the page. 

«^ Very positive reputation (website) 


pes puces ema 


This в an article on a newspaper website that 
has won a variety of awards. 


тһе arco has a comprehensive amount of 
very high quality MC that is unique and original. 
Including in-depth investigative reporting by two 
authors wih extensive knowledge and 
‘experience in investigative joumalism. 





Highest informational 1 





Fact-checking page on an 
Internet rumor debunking website 
(мү) 








+ А satisfying or comprehensive amount of 
very high quality MC, 

© Very high level of E-A-T for the purpose 
of he page 

= Positive reputation (website) 





tet 


= 


This page is on a wellknown fact-checking and 
debunking website that covers urban legends 
Tntomet rumors, and other stories of unknown 
ог questionable origin. 


The page has a very satisfying amount of 
original MC for users seeking fo validate a 
am about a documentary fim about 
mermaids. Users can rust the information on 
this page due to the website's postive 
reputation and high leval of expertise in 
debunking stories of his type. 
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Wobpage/Type of Content | Highest Quality Characteristics PQ Rating and Explanation 
و‎ 


Highest: informational 2 


Ball gown wedding dross page 


© A satisfying or comprehensive amount of 
very high quality МС 

‘© Vary high evel of E-A-T for tho purpose 
ofthe page 

= Positive reputation (website) 


‘This page has a vary satisfying amount of MC 
{or users interested in ball gown wedding 
dresses. An abundance of pictures, plus 
‘options fo view by price range, style, etc., are 
part of what makes this page so satisfying. 
This page is on a very popular wedding 
planning website. 




















مبب 
Very high evel of EAT for he purpose | ™™ hs‏ = 
Honest Govemment agency‏ 
ofthe page ‘This page has а ight зау amount ot‏ 
Yosemite Nationa Pak * Asatsiyng or comprehensive amount teil fomaton about Yosemite National‏ 
intonation ver high qually MC for the purpose ot | Pay including trafie alerts and inks other‏ 
мию pants of the website wih addtional йонтайоп.‏ 
Since the page is оп tne fit natora park‏ 
website, tis uniquely autora,‏ 
مبب 
Lene Nesim Hinen‏ 
Tn + Yay Noh valot EAT or be purpose‏ 
a othe page “This page otters а satistying amount of helpful‏ 
Sotware tps + A satstying or comprehensive amount of | ang very high quay МС for Ine purpose of he‏ 
кеч age. Because Ine pags is on he оа‏ 
Website ol he company tat produces the‏ 
software, is highly authoritative.‏ 
а‏ 
From tis Wkipatla article, we leam that‏ 
“Karalar i а main repository of source code‏ 
Highest Technical 2 * Very gh level o E-A-T for the рирове çor the Linux kernel the base of Ie popular‏ 
Se ae proba amount of | LUX operating sysler. makes al versions‏ 
Asalstyingox comprehensive amount o! cf tne source code aval о вї users.‏ = 
Linux Kerel archives Er fio source code aval о al users. The‏ 


‘main purpose cf the site is to host a repository 
for Linux kernal developers and maintainers of 
nux estbutions.” 


“This website is the most authoritative source for 
information about Linux. 








Highest Financial 1 


Credit raport information website 
(мт). 





Very high level of E-A-T for the purpose 
ofthe page 
‘© Very positive reputation (website) 





[inen aS EN 


Users in the U.S. can obtain free credit reports 
оп this website by providing their Social 
‘Security Number. This Wikipedia aricie tals us 
‘that this website is "the only federally mandated 
and authorized source for obtaining а free 
Credit report” 
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Wobpage/Type of Content — | Highest Quality Characteristics PQ Rating and Explanation 
тн etna 


Highest: Financial 2 


Tax forms page (YMYL) 


‘© Very high level of E-A-T for tho purpose 
ofthe page 

e А satisfying or comprehensive amount of 
very high quality МС far the purpose of 
the page 


The purpose of the page is to provide income 
tax forms and publications. The website 
represents an agency of the U.S. federal 
‘government that handles taxes, Бо the page 
and forms are uniquely authortatve and 
trustworthy. 





Highest: Magazine Aricie 1 


Алде tiled “Secret Fears of he 
Бароят 


‘© Very high level of E-A-T for tho purpose 
ofthe page 

‘© А satisfying or comprehensive amount of 
very high quality MC 

© Very positive reputation (website) 


mpm 


- 
Highest 


This article is on a magazine website that has 
won multiple National Magazine awards. The 
апе has a comprehensive amount of very 
gh quality and in-depth MC that is unique and. 
origina 





Highest: Magazine Aricle 2 


Interview with musical artist 


e Very high level of E-A-T for tho purpose 
ofthe page 

* А айсу or comprehensive amount of 
very high quality MC. 

‘© Very positive reputation (website) 


feather 


Lovee manum 


— 


ses 


This magazine в very wellknown and well 
regarded for content about artists and bands, 
‘and врвойсайу has a positive reputation for 
interviews with musicians. This article features 
high aualty and original content in its interview 
with the arte. 





Highest: Medical 1 


BMI calculator (YMYL) 


© Vary high level of E-A-T for the purpose 
ofthe page 
e Very positive reputation (website) 


Jus passen pen 


Lorem Nesim Hee 


The purpose of this page is to provide an online 
BMI (Body Mass Index) calculator. The 
calculator is functional and easy to use. 


Tho website roprosanls an institute that is part 
‘ofthe National Institutes of Health, It has an 
‘extremely good reputation and is an expert in 
medical topes. 





Highest: Medical 2 
Page about meningitis (YMYL) 


= Vary high level of E-A-T for the purpose 
of he page 
+ Very positive reputation (website) 


This в a meningitis reference paga on a 
trustworthy and authoritative website far a 
nonprofit medical research group. This website 
has a reputation of being one of the best web 
resources for medical information 











Highest: Medical 3 


Page about seasonal lu (YMYL) 





© Vary high level of E-A-T for the purpose 
ofthe page 
© Vary positive reputation (website) 





pept е 
Loen m are 


This is an influenza reference page on a 
trustworthy and authoritative medical website. 
This website has a reputation of being one of 
the best web resources for medical information 


ofthis type. 
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Wobpage/Type of Content | Highest Quality Characteristics PQ Rating and Explanation 
Jap pase ons 


Highest: Medical 4 


Health information on а hospital 
page (YMYL) 


= Very high level of E-A-T for tho purpose 
ofthe page 


‘According to this рада анда. this hospiat 
s "currently regarded as one of the top 4 
hospitals inthe United States as rated by L.S. 


News & World Варо" Users can trust 
mecical information on this website. 





© Vary high level of E-A-T for the purpose 


dope 
kowt Medium Agen 


“The purpose о! this page is to provide 
information about, and alow users to buy, a 





m— e rge 
oft page Pes ol fell rac рита, эв 
ИОД + Ва rp oman) | ene aot ah 
[т трнд 3 Aenityngorconpreensive amount of 
Nova VC ec he cro pedcs tis baia, hy 
дерек отба mice танар iti 
ка канын rrr 
sortie dreht rcg се Г Же 
orant ra mos раш od 
ДАЕ 
А satisfy it af high quality МС > ы ! — 
eem ур ora Mc | аа gta 


Onine banking login page 
мү) 


for ts purpose (login page) 

= Positive reputation (website) 

$ Very high level of E-A-T for the purpose 
ofthe page 


This page has login functionality, as well as 
daar information about what the user is logging 
into. This is a large. popular bank with a good. 
iutaton. 








Highest C 





Homepage ofa charity (YMYL) 


‘© Very high level of E-A-T for tho purpose 
ofthe page 
‘© Very positive reputation (website) 


- 


This в а highly reputable charity according to 
multiple chariy rating organizations. 








Highest Page with Error 
Message 


"Custom 404° page 





* ^ A satistying amount of high quality МС 
for ts purpose (error page) 

© Vary positive reputation (website) 

$ Vary high leval of E-A-T forthe purpose 
ofthe page 





ا 


This в an example of a "custom 404” page. 
These pages are designed to alert users that 

the URL they аге trying to visit no longer exist 
The MC of this page is the cartoon, tho caption, 
and the search functionality, which is specific to 
е content of the website. it is clear tha ime 

ftrt and talant were involved in ts creation. 





This publication has а very positive reputation. 
andis spacicaly known for its cartoons. 


Keep in mind that for any type of page, 
including pagas with error messages, there 
may be а range of highest quality to lowest 
aualty. Therefore rs important to evaluate the 
Page using the same criaria as all other pages, 
regardless of what type of paga t is. 
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Wobpage/Type of Content | Highest Quality Characteristics PQ Rating and Explanation 
ووا‎ 
Lort Medium Het 


= A satisfying or comprehensive amount of 








Highest: PDE Ele 
very high quality MC 
B + Very nign level oT E-A-T forthe purpose | major university, Which s hosted on o oficial 
ampus map Фере ors орыш. This i a Nolo euorabvo 
ura tor ls formation. Tha map dudes a 
sting of al me bulangs, parking structures. 
parking lots. construction areas etc. 
See 
^ Very high level of ЕА forthe purpose |The website is one af ihe most popular recipe 
наза Racines 1 
othe page webstes and be auihor of he recipe s a 
« WesleipamcunofhghquayMC | wek anaa celebrity chat who spatiales n 
Pumpkin pia recipe permei AAMO co be 


This page has a large number of user 
ot me recipe, detailed preparation time 
information, a "recipe box" feature, ele. 








Highest: Recipes 2 


(Chocolate chip cookie recipe on a 
recipe blog 


‘© Very high level of E-A-T for tho purpose 
ofthe page 

© А satisfying amount of high quality MC 
‘nd features 


_—+————+—=ң 


Even though this usar does nol seam to be а 
‘wol-known professional chef, recipes aro an 
‘example of everyday expertise. The author of 
this blog has documented her extensive. 
‘experimentation witha chocolate chip cookie 

ipe, and her expertise is demonstrated in the 
large quantity of anginal high or highest quality 
wc. 








Highest: Video 1 


The band OK Go's music video 
for the song "This Too Shall Pass" 


= Авайыўпа or comprehensive amount of 
very high quailty MC. 

Very high level of E-A-T for the purpose 
ofthe page 


- 


This is a high qualty professionally produced 
Video on the band's oficial channel. The music 
Video represents unique and original content 
created by a highly skilled and talented musical 
arat 








Highest Video 2 


Henri 2, Paw de Deu video 





‘© А satisfying or comprehensive amount of 
very high quality MC 

=. Very positive reputation forthe content of. 
this particular video 





ат 


= 


This is a high quality, popular video. It won the 
"Golden Kitiy Award for “Best Cat Video On. 
The internet at tne Walker Art Canter Internet 
Cat Video Fim Festival. The video is unique 
and original content ereated with a high degree 
af lime and efr 
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Wobpage/Type of Content | Highest Quality Characteristics PQ Rating and Explanation 
pompes so 


Highest: Vien 3 


Saturday Night Live video on the 
TV networks website 


* A satisfying or comprehensive amount of. 
very high quality МС 

‘© Vary high evel of E-A-T for tho purpose 
ofthe page 


The MC of this video page в an episode ог 
Saturday Night Live on the TV network's official 
‘website, making it highly authoritative and 
trustworthy. The episode represents high 
quality and original contant created by a TV 
how that has won numerous awards, Below 
the main video here ara many other videos 
that users may be interested in. 





Highest: QA. 


QRA page on abbreviations 


© Vary high level of E-A-T for the purpose 
ofthe page 
© Vary positiva reputation (website) 


CAN O em pomi 


The purpose of the page is to answer questions 
about the usage of abbreviations. This website 
has a very good reputation as a reference for 
information on writing, publishing, et. Itis 
Considered highly authoritative and trustworthy 
for the topic of the page. 





Highest: Social Madia Profle 


The oficial Twitter profi page of 
tha Tennessee Republican Party 
ove) 





Very high level of E-A-T for the purpose 
ofthe page 

= А satisfying or comprehensive amount of 
high quality MC 





ptt 


- 


This is the official Twiter account for the 
Tennessee Republican Party (TNGOP). It is an 
expert and authoritatve source of information 
about the Tennessee Republican Party. 
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6.0 Low Quality Pages 


Low quality pages may have been intended to serve a beneficial purpose. However, Low quality pages do not achieve 
their purpose well because they are lacking in an important dimension, such as having an unsatisfying amount of MC, ог 
because the creator of the MC lacks expertise for the purpose of the page. 


1га page has one or more of the following characteristics, the Low rating applies: 


‘An inadequate level of Expertise, Authontativeness, and Trustworthiness (E-A-T). 

The quality of the MC is low. 

There is an unsatisfying amount of MC for the purpose of the page. 

The title of the MC is exaggerated or shocking. 

The Ads or SC distracts from the MC. 

There is an unsatisfying amount of website information or information about the creator of the MC for the purpose 
of the page (по good reason for anonymity). 

« A mildly negative reputation for a website ог creator of the MC, based on extensive reputation research 





Ifa page has multipla Low quality attributes, a rating lower than Low may be appropriate. 


6.1 Lacking Expertise, Authoritativeness, or Trustworthiness (E-A-T) 





Low quality pages often lack an appropriate level of E-A-T for the purpose of the page. Here are some examples: 

‘© The creator of the MC does not have adequate expertise in the topic of the MC, e.g. a tax form instruction video 
made by someone with na clear expertise in tax preparation. 

‘© The website is not an authoritative source for the topic of the page, e.g. tax information on a cooking website. 

©The MC is not trustworthy, e.g. a shopping checkout page that has an insecure connection, 


Note: Websites with user-generated content span the Page Quality rating spectrum. Please pay careful attention to 
websites that allow users to publish content with litle oversight, such as social networking pages, video sharing websites, 
wolunteer-created encyclopedias, article sharing websites, forums, etc. Depending on the topic, pages on these websites 
may lack E-A-T. 


Important: The Low rating should be used if the page lacks appropriate E-A-T for its purpose. 


62 Low Quality Main Content 


The quality of the MC is an important consideration for РО rating. We will consider content to be Low quality if itis 
created without adequate time, effort, expertise, or talentiskill. Pages with low quality MC do not achieve their purpose 
well 


In addition, please examine the title on the page. The tile of the page should describe the content. 


Exaggerated or shocking tiles can entice users to cick on pages in search results. If pages do not live up to the 
‘exaggerated or shocking title or images, the experience leaves users feeling surprised and confused. Here is an example 
of a page with an exaggerated and shocking title: "ls the World about to End? Mysterious Sightings of 25ft Sea Serpents 
Prompt Panic!” as the tite for an article about the unidentified remains of one small dead fish on a beach. Pages with 
‘exaggerated or shocking titles that do not describe the MC well should be rated Low. 





Important: The Low rating should be used if the page has Low quality MC. 
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63 — Unsatistying Amount of Main Content 


‘Some Low quality pages are unsatisfying because they have a small amount of MC for the purpose of the page. For 
‘example, imagine an encyclopedia article on a very broad topic such as World War Il that has just a few paragraphs. 


Important: The Low rating should be used if the page has an unsatisfying amount of MC for the purpose of the page. 


64 Distracting Ads/SC 


We expect Ads and SC to be visible. However, some Ads, SC, or interstitial pages (Le., pages displayed before or after 
the content you are expecting) make it difficult to use the MC. Pages with Ads, SC, or other features that distract from ог 
interrupt the use of the MC should be given a Low rating 


A single pop-over Ad or interstitial page with a clear and easy-to-use close button is not terribly distracting, though may 
not be a great user experience. However, difficulo-ciose Ads that follow page scrolls, or interstitial pages that require an 
арр download, can be truly distracting and make the MC difficult to use. You can see examples of interstitial pages here. 


The content of the Ads, SC, or other features тау be distracting as wel: sexually suggestive images such as here, 
grotesque images such as here, and рот Ads on поп-рот pages should be considered very distracting. 


Finally, Ads and SC can be distracting if the tiles or images of the Ads or SC are shocking or disturbing. Here is an 
‘example of a page with shocking and exaggerated titles, images, and text in the Ads and SC. 


Important: The Low rating should be used if the page has Ads, SC, or other features that interrupt or distract from using 
the NC. 


6.5 Mixed or Mildly Negative Reputation of the Website or Creator of the Main Content 


Extensive reputation research is required for ай РО rating tasks unless you have previously researched the reputation of 
the website. Many websites have litle reputation information, unfortunately. Of the websites with reputation information, 
most websites have a good reputation. Please exercise care when researching the reputation of businesses. Try to find 
as many reviews and ratings as possible, and read the details of negative reviews and low ratings before inferring that the 
business overall has a negative reputation. A few negative customer service reviews are typical for businesses such as 
restaurants, 


Ifthe MC was not created by the website, research the reputation of the creator of the МС. While many ordinary people 
do not have reputation information available on the Intemet, you can find reputation information on well-known YouTubers, 
joumalists, authors, bloggers and vloggers, professionals such as lawyers and doctors, ete. 

Pay attention when there is evidence of mixed or mildly negative—though not malicious or financially 
fraudulent—reputation, The Low rating should be used if the website or the creator of the МС has a mildly negative 
reputation. 


Important: Fora YMYL website, a mixed reputation is cause for a Low rating 
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6.6 — Unsatisfying Amount of Information about the Website or Creator of the Main Content 


We expect some form of website information for many or most websites. We expect clear information about who (e.g. 
what individual, company, business, foundation, etc.) created the MC, unless there is good reason for anonymity. А 
long-standing Internet alias or username can also serve the same function as identifying the MC creator. However, the 
amount of information needed about the website or creator of the МС depends on the purpose of the page. For personal 
websites or non-YMYL forum discussions, an email address or social media link alone тау be sufficient. 


‘Stores and websites that process financial transactions require a high level of user rust. If a store or financial transaction. 
website has just an email address and physical address, it may be difficult to get help if there are issues with the. 
transaction. Likewise, many other types of YMYL websites also require a high degree of user trust 


Important: For YMYL pages and other pages that require a high level of user trust, an unsatisfying amount of any of the 
following is a reason to give a page a Lew quality rating: customer service information, contact information, information. 
about who is responsible for the website or information about who created the content. Far other types of websites, use 
your judgment. 


67 Examples of Low Quality Pages 











TypeofWebpage/Content — | Low Quality Characteristics PQ Rating and Explanation 
bmp 
Low: News 1 The writing of this arici is unprofessional, 
+ Low quality MC. Including many grammar and punctuation 
Aricie about nuclear power ® Lacing EAT огиз. The МС also appears to have been 
ML) paraphrased from a since 2122 found on а 


‘ferent source, but with factual inaccuracies 
introduced throughout. The combination of low 
‘quality MC and lack of E-A-T justify he 
ovest to Low rating. 





Juice н 


Loses Meme chen 


The ita of the article is misleading and does 
not reflect he actual content ofthe page. The. 





Low: News 2 кыйак МС does not explain how to gel rid of toxins or 
e Low quay what the word “toxins in the headline refers lo. 

Article ted "Getting Rid Of = Misleading page tite 

Toxins After The Holiday Season" | 5 Lacking EAT айй olo has mai probent: the 

Eh ‘The content also has many problems: th 


ating quality is poor, and the article includes 
‘meaningless statements such as “watar 
therapy s one of ihe easiest ways of beauty 
regimen since it wil give enough moisture оп 
ihe skin". Tha article fails to cite sources, and 
there is no evidence of EAT. 
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Туре of Webpage/Contont 


Low Quality Characteristics 


PQ Rating and Explanation 





Low: nfarmational 1 
AI birthday party food ideas 


+ Low quality MC 
$ Disruptive Ads 


=ч 





This page and website have many 
characteristics of Low quality pages. Close 
lservation shows MC that contains тову 
‘commonly known information and poor quality 
wring. The МС is broken up by large Аав that 
аат the user experiance, 





Low informational 2 


© Unsaistying amount of MC for the 
Purpose of tna page 


pocas pnl 


мш medun teen 


Some websites rely оп users to creata virtual 





ОВА uniniierid question al oftheir MC. In ti case, he MC is the 
user's question. If here are no answers, he 
amount of MC on the page is unsatisfying. 

el 

[Low Aricie 1 م‎ Mes E 


Arie with tips for dressing for 
the office 


+ Low or Lowest quality MC. 
® Lacing EAT 


‘This content has many problems: 
inaccurate/mearingless information and 
‘complete lack of editing with poor spelling and 
grammar Бо of these characteristics in 
‘combination justify the Lowest to Low rating. 





Low: Arica 2 


Using ginger for your health 
ему 


e Low or Lowest quality MC 
® Lacing EAT 


pat 
мешт ges 





This content has many problems: garbled 
information and a lack of editing. Bath of these 
characteristics in combination just he 
‘Lowest to Low rating. In addition, lacking 
Е-АТ on this topic is concerning. 








Lou Arica a 


How to adapt children from Iraq 
vw) 





* ^ Low quality MC: commonly known 
information only 
© Lacking EAT on a YMYL tapie 





peras] sepe 


This is a YMYL topic. However, the stops isted 
here are merely commanly-known information 
that would be o tle benefi to someone 


interested in adopting a child from Iraq. 





For example, step 1 says "Choose an adoption 
agency and suggests looking in a phone book. 
There is almost no information specife to Iraq. 
There is no evidence of expertise on adaption 
from the description about the author. 


Note that some Ads have the same format as 
links to ather articles on this website, potentially 
making it a bit harder for users to visually 
distinguish these Ads from the MC and SC. 
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Туре of WebpageiContent — | Low Quality Characteristics PQ Rating and Explanation 
freee 
Low: Financial мш medun о нш 


Page with advice on picking a 
quality stock for Investment 
(мү), 


= Lacking EAT on a YMYL оре 


There is по evidence that the author has 
financial expertise. Because this is a YMYL 
financial article, lacking expertise is a reason 
fora Low rating. 





Low: Medical 


Page with information about how 
fang the fu lasts (YMYL) 


Lacking E-A-T on a YMYL topic. 


ا 
Mesem‏ 


There is no evidence that the author has 
medical expertise. Because tnis is a YMYL 
medical article, lacking expertise is a reason for 
= Low rating. 








Low: Recipes 1 


Apple pie recipe 


© Low quality and unsatisfying amount of 
мс 





This page and website have many of 
characteristics of Low qualy pages. The MC 
is low quality and lacks important information. 
For example, it gives no indication af how to 
make a crust and doesn't list premade crust as 
ап ingredient. It wouid be very диясы! to 
actually make an apple pie using this recipe. 


This page is lacking е kind of helpful content 
We expect in recipe pages, such as user 
eviews or comments, Without such 
information, its hard to tell if the recipe is any 
‘good. Tho small pink tex at the top is not 
helpful far users. 





Low: Recipes 2 


Gluten-free New York cheesecake 
recipe 


‘© Very distracting SC or Ads (not clear) in 
the пам colum. 
© Disruptive Ads in the middle column. 


D 





‘This в an example of distracting and disruptive 
AdsISC. there are large Ads in the main column. 
pushing down the MC as wall as highly 
distracting content on the righi that are labeled 
"тор Posts & Pages.” Itis unclear whether 
these are SC or Ads. 








Low: ават 


Page about Native American 
customs (YMYL) 





+ Lacu EAT 





SS 


There are 94 answers to this question with а 
few results that seem helpful. Many of the 
posts appear to be deliberately incorrect or. 
misleading, including the top answer, which is 
labeled the "nest answer” 
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Туре of Webpage/Contont 


Low Quality Characteristics 


PQ Rating and Explanation 





Low: 842 


Page about a 2002 Volvo part 
(мү), 


^ Misleading Ads 
5 Unsalishying amount of MC for the 
Purpose ol he page 


pomme pt 
ro vm 


Please read the MC (areas with rad boxes 
around i), including the completely unhelpful 
answer? This answer is so unhelpful, we can 
consider this question to be unanswered. This 
Page has an unsatisfying amount of MC. 


In addition о a very unhelpful “answer the 
pago design makes i difficut to distinguish the 
МС from Ads. For example, below the answer, 
же see a "sponsored answer” which has the 
‘same format as the real answer, but is actually 
an Ad and not an answer to the question ће 
s misleading to users. 








Low ORA 


Раде about water and зой 








= Misleading AdsiSc 
‘© Unsatistying amount of MC for the 
purpose of he page 





rm 


This is a page trom a Q&A sio with an 
unanswered question. Q&A pages exist to 
answer user questions. Pages with 
unanswered questions should generally be 
‘ated Low because they have litle MC (just a 
‘question and по answer) and don't achieve 
‘their purpose well 


In addition о having no answer, this page has 
‘Ads and links lo other questions (misleadingly 
labeled as "Relevant answers") displayed 
prominently, which users may mistake or. 
answers lo ihe question. I lakes a moment to 
notice that this page actually has no answer. 
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7.0 Lowest Quality Pages 
This section describes Lowest quality pages. 
As a reminder, here are the steps for doing PQ rating: 


1. Assess the true purpose of the page. If the website or page has a harmful purpose or is designed to deceive 
people about its true purpose, it should be rated Lowest. 

2. Assess the potential of the page to cause harm as described in these guidelines. Websites or pages that are 
harmful to people or society, untrustworthy, or spammy as specified in these guidelines should receive the 
Lowest rating. 

3. Otherwise, the РО rating is based on how well the page achieves its purpose using the criteria outlined in these 
guidelines. 


Below is a summary of different types of Lowest pages. Although listed separately, the categories below may overlap. 
(e.g. pages that contain Harmfully Misleading Information are often Untrustworthy as well) 





туре of Lowest Page | Description 





Harmful to Self or Other | Pages that encourage, depict, incite or directly cause physical, mental, emotional or 
Individuals financial harm to seif or other individuals. 


Examples of Harmful to Seif include: 
Online scams to steal personal information 
* Detailed instructions on how to commit suicide 





Examples of Harmful to Other Individuals include: 
• Doing (ie. publicly sharing a person's sensitive personal identification 
information) 
* Detailed, realistic and serious instructions writen with the intent to help 
someone commit murder. 





Harmful to Specified | Pages that promote, condone, or incite violence or hatred against a Specified Group of 
Groups people (as defined in Section 7.2). 


Examples include: 
e Content that encourages violence or il treatment towards a Specified Group 
* Content with extremely offensive/dehumanizing stereotypes of a Specified 
Group 





Harmfully Misleading | Pages that misinform people in a way that could cause harm 
Information. 
Examples include: 

‘© Clearly inaccurate harmful information that can easily be refuted by 

straightforward and widely accepted facts 

Harmful information that contradicts wel-established expert consensus. 
Harmful unsubstantiated theories/claims not grounded in any reasonable facts 
or evidence. 
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туре of Lowest Page | Description 





Untrustuorthy Pages or websites that are deceptive or have untrustworthy characteristics 
Webpages 
Examples include: 
< Pages or websites with deceptive purpose or design 
< Pages or websites you strongly suspect are engaged in malicious behavior 





‘Spammy Webpages ^ | Pages with characteristics of webspam as defined in the Google Search Central 
Webmaster Quality Guidelines and/or Section 7.5 of this guideline 


Examples include: 
* Pages deliberately created with no MC or gibberish MC 
e Hacked, defaced, or spammed pages. 














Noto: There is lots of content on the Internat that some would find controversial, one-sided, off-putting, or distasteful, yet 
would not be considered Lowest based on these guidelines. Please use your judgment to follow the standards outlined 
throughout Section 7.0 rather than relying on personal opinions, 


7.1 — Harmful to Self and Other Individuals 


Use the Lowest rating for pages with content that encourage, depict, incite, or directly cause harm to self or other 
individuals. 


Harm includes physical, mental, emotional, or financial harm to people. Pages should be considered Harmful to Self or 
Other Individuals if they directly attempt to harm people; encourage behavior that may result in harm; depict extremely 
violent or gory content without a beneficialeducational purpose; or otherwise are severely traumatic to people who view 
the page. 


Pages do not have to be harmful to all people to be considered Harmful to Self or Other Individuals. Different people have 
diferent levels of vulnerability to scams, awareness of potential dangers (e.g., dangerous feats depicted in stunt videos), 
and tolerance for viewing violentidisturbing content. If there is a reasonable possibility that viewing a particular page 
would cause harm to these who are most vulnerable, it should be considered harmful. 


Pages created with a beneficial purpose that report on, discuss, ог inform about harmful actions or events (e.g. fictional 
entertainment, reputable news, education) should typically not be considered Harmful to Self or Other Individuals. For 
‘example, advocacy aimed at drawing attention to harmful, real-world actions or events (such as a webpage describing a 
protest against domestic violence) would not be considered Harmful to Self or Other Individuals. 


Examples of pages that are Harmful to Self or Other Individuals include any of the following types of content: 

‘© Content containing serious death threats or other realistic-sounding threats 

‘© Content that shares personal information belonging to others with malicious intent to target them or promote 
harassment towards them (Le.,"doxxing’) 

© How-to or step-by-step information that describes how to commit violent acts in an easy-to-replicate way 

Content meant to advocate for, glorify, or trivialize violence and atrocities, or to disparage victims) of 
violenca/atrocities 

«Content depicting or promoting information that facilitates or leads to serious harm to people, or discussions that 
attempt to justify abuse of people 

‘© Content that encourages unsafe behavior or substantially downplays the risks of dangerous activities (e.g. 
consuming household cleaning products) 

‘© Suicide promotion or pro-anorexia content that encourages people to engage in behavior that can result in 
hospitalization or death 
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‘© Health-related advice that contradicts well-established expert consensus and could result in serious harm (e... 
statements that lemons cure cancer), or could prevent someone from undertaking a life-saving treatment (e.g. 
encouraging a home remedy as a replacement for standard medical treatment to cure a disease) 

‘© Malicious pages that attempt to scam or hurt people, contain suspicious links (e.g., to download malware), 
request personal information without a legitimate reason, or “phish” for passwords 


Examples of content that should not be considered Harmful to Self or Other Individuals include: 
© Depictions of violence in an action movie 
А news story about violent events 
Educational content that may depict violence or gross imagery 
An explanation of scams meant to raise awareness about them 
Portrayals of dangerous activities in a manner that discourages others from attempting the same feat (such as by 
clearly explaining the risks involved, describing the expertise and equipment required, etc.) 


72 — Harmful to Specified Groups 
Use the Lowest rating for pages that promote, condone, or incite hatred against a Specified Group of people. 


For the purposes cf Search Quality rating, a Specified Group is a group of people that can be defined on the basis of: 
* Age (e... older adults) 
Caste (e.g. Dalits) 
Disability (e.o.. people who are blind) 
Ethnicity (e.g. Roma) 
Gender Identity and Expression (e.g, transgender people) 
Immigration Status (e.g. student visa holders) 
Nationality (e.g. Italians) 
Race (e.g., Asians) 
Religion (e.g, Christians) 
Sex/Gender (e.g. men) 
Sexual Orientation (e.g., lesbians) 
Veteran Status (e.g., Marines) 
Victims of a major violent event and their kin (.9.. vitcims of the Holocaust) 
Any other characteristic that is associated with systemic discrimination or marginalization (e.g. refugees, people 
‘experiencing homelessness) 





Examples of pages that are Harmful to Specified Groups include any pages containing content that: 
© Encourages violence or il treatment towards a Specified Group 
Promotes intolerance by demonstrating a staunch unwillingness to allow for the views, beliefs, or behavior of a 
Specified Group 
‘© implies that one Specified Group is superior or inferior to another 
‘© Contains extremely offensive/dehumanizing stereotypes of a Specified Group. Note that stereotypes can be 
negative or positive. 


The tone of the harmful content must be either serious (e. not joking or satirical) ог mean-spirited (.е. with an intent to 
demean or promote intolerance) to be considered Harmful to Specified Groups. Satirical comedy or artistic expression 
related to a Specified Group should not be considered harmful unless it is clearly mean-spirited or has other clear harmful 
impact 


Criticism of objects, philosophies, and ideas are generally not considered Harmful to Specified Groups. For example, 
negative criticism af a religious doctrine should not be considered targeted at the Specified Group that follows that 
religion. Remember that the content must promote, condone, or incite hatred of people to be considered Harmful to 
Specified Groups. 


Educational pages (e.g., definitions, research, academic papers), news stories, or other pages that have a beneficial 
purpose of informing society should not be considered Harmful to Specified Groups. Similarly, historical 
documentsivideos that aim to capture the beliefs of different eras should not be considered Harmful to Specified Groups. 
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Examples of content that should not be considered Harmful to Specified Groups include: 


34 


A historical documentary of WWII featuring speeches from Nazi leaders 
A stand-up comedy routine that plays off of stereotypes in a way that is not mean-spirited 
А newspaper article about a hate organization 

The dictionary definition of a slur 

A discussion about a particular religious text and its views on women 


Harmfully Misleading Information 


Use the Lowest rating for pages that mislead people in ways that can cause ham to people and society. 


Misleading pages may have been produced with the intent to misinform people, or the content creator may believe that 
the inaccurate information thay are sharing is true. There is an especially high standard for accuracy on YMYL topics or 
other topics that can cause harm. Be sure to research consequential facts or claims as necessary and to the extent the 
task time allows. 


Pages should be considered to contain Harmfully Misleading Information when they contain at least one of the following: 


Harmful and clearly inaccurate information that can easily be refuted by straightforward and widely accepted 
facis (e.g. false claims that a worid leader has died, misleading or false statistics оп gun violence, еіс.) 

Harmful claims that contradicts well-established expert consensus (e.g., lemons cure cancer), with expert 
Consensus defined as a set of positions, facts, or findings that are widely agreed upon by authorities in the 
relevant feld (e.g. widely-adopted medical guidelines, an investigative report put forth by a relevant watchdog 
group, etc.) 

Harmful unsubstantiated thoories/clalms not grounded in any reasonable facts or evidence, especially those 
that could erode confidence in public institutions. This includes unsubstantiated theories that have either been 
thoroughly debunked (e.g. the 9/11 attacks were planned by the United States government) or are too outlandish 
to be given credence (e.g., several world leaders are lizard people). 


However, note that some types of information are subjective, debatable, unverifiable, or inconsequential. For example, 
pages should not be considered to contain Harmfuly Misleading Information if they exclusively contain: 


Non-YMYL content created with a clear entertainment purpose, containing no hard claims of factual accuracy and 
are not harmful to people or society. Examples include many types of fiction, satire or parody, astrology, folklore, 
myths, and urban legends. 

Reviews expressing personal preferences, opinions, or value-based judgments about a product, restaurant, 
bbookimovie/TV show, etc. 

Claims or statements that are reasonably debatable when there is not a single established correct answer or truth 
(e.g. discussions about the relative effectiveness of diferent healthcare systems) 

Insignificant errors or inaccurate information about a trivial topic (e.g., inaccuracies in the height of a celebrity) 


Pages that aim to persuade others that a certain position or perspective is correct are fairly common on the Internet. 
Pages with one-sidediopinionatedicontroversialipolarzing content should not be considered to contain Harmfully 
‘Misleading Information unless they could create harm to individuals or Specified Groups (as described above) and contain 
clearly inaccurate information, contradict well-established expert consensus, or are not grounded by reasonable 
factsievidence. 


A webpage may be considered to have Harmfully Misleading Information based on the MC or on other characteristics of 
the page or information about the content creator (e.g., the lle of an article is harmfully misleading, even if the article оп 
its own is not; a creator blatantly misrepresenting their medical credentials for a video on medical topics). 
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Finally, note that Harmfully Misleading Information can be especially hard to identify because it may require research from 
outside sources. Reputable fact-checking websites can't always keep up with the volume of unsubstantiated 
theorias/claims produced by the Internet, and some theories may even claim that debunking information is inaccurate. 
You should attempt to find high-quality, trustworthy sources to check accuracy and seek out the consensus of experts if 
you are unsure. Please research theories and claims to the extent the task time allows. If a theory/claim seems wildly 
improbable and can't be verified by independent trustworthy sources, you should consider it unsubstantiated 


74 — Untrustworthy Webpages or Websites 


The Lowest rating should be used for pages or websites you strongly suspect are engaging in deceptive or malicious 
practices. 


Because some deceptive pages may in fact be harmful, please use caution. Known or "obvious" scams are clearly 
harmful and should be rated Lowest, but there may also be pages you strongly suspect are scams yet cannot prove it 
without experiencing harm yourself. Please consider these pages to be untrustworthy and use the Lowest rating 


Your assessment of untrustworthiness may be based on the content of the page, information about the website, 
information about the content creator, and the reputation of the website or content creator. 


Your assessment may also be based on a lack of critically important information. For example, any website involving 
financial transactions or sensitive information should have comprehensive information about who is responsible for the 
site and a way to contact tho site if something goes wrong. 


If some aspect of a page or website makes you suspect deception or maliciousness, please look for information about the 
site. If you cannot find reputation information to confirm your suspicion, carefully explore the site. Sometimes a single 
page on an unknown website in isolation may seem odd but subsequent exploration shows no concern. However, if you 
see a pattern of what appears to be deception or manipulation or become concerned about your own safety, please use 
the Lowest rating and leave the website Immediately. 


Pages with the following characteristics should be considered Untrustworthy: 
‘© Inadequate information about the website for its purpose 
Lowest E-A-T or Lowest reputation 
Deceptive purpose, deceptive design, or deceptive intent. 
Deliberately obstructed or obscured MC 
‘Suspected malicious behavior 


Important: Highly untrustworthy pages should be given the Lowest rating even if you are unable to "prove" the webpage 
or site is harmful. Because many people are unwiling to use a highly untrustworthy page, an untrustworthy page or 
website fails to achieve its purpose. 


74.1 Inadequate Information about the Website or Creator of the MC for the Purpose of the Page 


Providing information about who created the content and who is responsible for the content is critical to building trust with 
people who visit the page. 





As discussed in Section 2.5.3, we expect most websites to have some information about who (e.g., what individual, 
‘company, business, foundation, etc.) is responsible for the website and who created the MC, as well as some contact. 
information, unless there is a good reason for anonymity. As you will see in the examples below, many types of Lowest 
pages such as malicious downloads and scams typically have no information or fake information about who is responsible. 
to prevent recourse by people who are harmed, 


For websites with YMYL pages, such as online banks, we expect to find a lot of information about the site, including. 
‘extensive customer service information. Any site that handles personal, private or sensitive data must have adequate 
Contact information. This includes sites that ask users to create passwords, share personal information, or conduct 
financial transactions 
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YMYL pages or websites that handle sensitive data with absolutely no information about the website ог creator of the MC 
should be rated Lowest. 


Think about the purpose of the page and the purpose of the website. What type of content creator or website information 
would people expect or demand in order to feel comfortable using the webpage? If the website or page lacks critical 
information about who is responsible for the content to the extent that it feels untrustworthy, it should be rated Lowest. 


7.4.2 Lowest E-A-T and Lowest Reputation of the Website or Creator of the MC 
Reputation and E-A-T are some of the most important considerations in Page Quality rating. 

Ifthe E-A-T of a page is low enough, people cannot or should not use the MC of the page. This is especially true of 
YMYL topics. If а YMYL page is highly inexpert or highly unauthortaive for its purpose, it should be considered 
Untrustworthy and rated Lowest. 


Recall that a page lacking adequate expertise or authoritativeness should be rated Low. Lowest is appropriate for pages 
that are so lacking in expertise or authoritativeness that they are untrustworthy. 


Use the Lowest rating if the website and the creators of the MC have an extremely negative reputation, to the extent that 
many people would consider the webpage or website untrustworthy. 


Here are some examples originally shown in Section 2.6.4. All pages on these websites should be rated Lowest because 
of the credible extremely negative or malicious reputation of these websites, making them untrustworthy. 


Noto: These website examples may be considered Harmful in some form as well. Sometimes reputation research 
‘exposes information that may not meet the criteria for considering a website harmful but does cause the website to be 
untrustworthy. 





Website Description. 





Evidence of fraudulent behavior: There are many reviews on websites describing how users 
‘Site salina children's jungle aym | sent maney and did not receive anything in retum. There в siso а news arci about hie 
business engaging in financial fraud. 


Site saling praducis related io | Evidence of criminal behavior: This website engaged in criminal behavior such as physically 
Eyewear threatening users. 








Organization serving the Evidence of fraudulent behavior. There are many detailed negative articles on news sites and 
Dosplalzed velaran community | charity watchdog йез describing fraud and financial mishandling about this organization 











74.3 Deceptive Purpose and Deceptive Page Design 


People who browse content online have a wide range of Intemet understanding and savviness. Consider a page to be 
leceptive" if it may deceive some people. All deceptive pages should be rated Lowest because pages that engage in 
deception are Untrustworthy. 


Pages or websites are Untrustworthy if they have a deceptive purpose. These pages or websites superficially appear to 
have one purpose, but in fact exist for a different reason. 


The following list includes some common types of deceptive purpose pages. However, no list of deceptive page types will 
be complete—deceptive websites continue to evolve as people and search engines figure out how they are being Wicked. 


‘© Awebpage or website that impersonates a different site (e.g, copied logo or branding of an unaffiliated site, URL 
that mimics another site's name, etc.) 

‘© Anon-satiical social network profile made by an impersonator 

< A webpage or website that looks like a news source or information page, but in fact has articles to manipulate 
users in order to benefit a person, business, goverment, or other organization politically, monetarily, or otherwise 
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‘© Awebpage claims to offer an independent review or share other information about a product, but is п fact created 
to make money for the owner of the website without attempting to help users. For example, the MC may contain 
intentionally misleading or inaccurate information created with the sole purpose of getting users to click on 
monetized links or buy the product 

‘© Awebsite claims to be the personal website of a celebrity, but the website is actually created to make money for 
the owner of the website without the permission of the celebrity. For example, the page may have false 
testimonials fora product and is created for the sole purpose of getting users to click on monetized links or buy 
the product 

‘© A webpage with a misleading title or а title that has nothing to do with the content on the page. People who come 
to the page expecting content related to the title wil feel tricked or deceived 

< Awebpage or website with deceptive website information. For example, the website may misrepresent who owns 
the site, what the website purpose really is, how the content was created, how to contact the site, et. 


Another form of deception is through the design of the page. Some pages are deliberately designed to manipulate users 
to take an action that will benefit the owner of the website rather than help the user. 


Here are some common types of deceptively designed pages: 

‘© Pages that disguise Ads as MC. Actual MC may be minimal or created to encourage users to click on the Ads. 
For example, fake search pages (example) that have a list of links that look like a page of search results. If you 
click on a few of the links, you wil see that the page is just a collection of Ads disguised as search engine results. 
A "search box" is present, but submitting a new query just gives you a diferent page of Ads disguised as search 
results. 

‘© Pages that disguise Ads as website navigation links. For example, fake directory pages (example) that look 
like a personally curated set of helpful links, possibly with unique descriptions. In reality, the links are Ads or inks 
to other similar pages on the site. Sometimes the descriptions of the links are unrelated to the page. 

‘© Апу page designed to trick users into clicking on links, which may be Ads or other links intended to serve the 
needs of the website rather than to the benefit of the user. 


Take a good look at the page and use your judgment. If you believe the true purpose of the page is not what it appears to 
be, the page should be rated Lowest. If you believe the page was deliberately created to manipulate users to click on 
Ads, monetized links, questionable download links, etc., rather than to help people, the page should be rated Lowest. If 
the page uses deception in any form even if not described here, the page should be rated Lowest. 


7.4.4 Deliberately Obstructed or Obscured MC 


Pages are untrustworthy if the MC is deliberately obstructed or obscured due to Ads, SC, interstitial pages, download links 
ог other content fat is beneficial to the website owner but not necessarily the website visitor. Attempts to manipulate or 
‘coerce users away from the MC is evidence of untrustworthiness, which makes the Lowest rating appropriate. 


Here are some examples of pages with deliberately obstructed or obscured MC that should be rated Lowest: 
‘© Ads that continue to cover the MC as the website visitor scrolls down the page. The Ads are virtually impossible 
to close without licking on the Ad 
Pop-ups that obscure the MC and cannot be closed without taking an action that benefits the website 
‘An interstitial page that attempts to coerce a download or ciick that does not benefit the website visitor 
‘Ads that push the MC down so far that many users would not notice the MC 
MC in white-on-white text or MC presented so that itis difficult for a human being to read 


Logins ог paywalls on otherwise trustworthy sites should not be considered untrustworthy, deceptive, or coercive. Many 
high quality sites such as newspapers or magazines cannot exist without monetization through subscriptions and 
paywalls. Some types of MC should be login protected, such as pages with personal information online banking websites 
or private pages on social network websites. 





МС may be inaccessible for non-deliberate or non-coercive reasons, such as content that doesn't load or content that is 
unavailable in a country or region. These are not reasons for Lowest. 


Use your judgment to assess whether the page is designed to deceive or coerce users into taking actions that benefit the 
website. Ifyou are unsure, look at other pages on the same website, conduct reputation research, tc. 
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Remember many websites need monetization to share content with users. The presence of Ads alone is not enough for 
Lowest. The important criteria for Lowest is deception, manipulation or other coercive attempts to get people to engage 
with monetization or Ads rather than the MC. 


7.4.5 Suspected Malicious Behavior 


"tis frequently dificult to prove that a page is malicious without experiencing harm. For this reason, you may use the 
Lowest rating if you strongly suspect a page is malicious or harmful even without having proof. Any of the following 
should be considered untrustworthy: 


‘= Pages or websites that you strongly suspect are scams (see these links for more information about Intemet 
scams: Avoid Scams 1, Avoid Scams 2, Internet Fraud). 

‘© Pages that ask for personal information without a legitimate reason (for example, pages that ask for name, 
birthdate, address, bank account, government ID number, їс. 

‘© Pages that appear to “phish” for passwords to Facebook, Gmail, or other popular online services. See here for 
information about “phishing” fraud. 

‘© Pages with suspicious links, including links you suspect are malware download links. 


Important: Do not continue to interact with the page if you observe these characteristics. 


If you do click on a link and encounter a malware warning from your browser or antivirus software (example), please do 
not continue to that page. You can assume that the page is malicious for the purpose of rating and use Lowest. 


Note that not ай warnings your browser may display are related to malware (e.g. other types of browser warnings include 
certificate acceptance requests, content filtering warnings, etc). If you are unsure or hesitant to continue to the page for 
апу reason, you can release the task. 


7.5 Spammy Webpages 


Remember that pages and websites should exist for the benefit of people who visit the website. There are many types of 
webpages that can benefit visitors, such as online shopping or banking services, videos offering entertainment, ог 
personal pages sharing a perspective or experience. Some of these pages depend on advertising and monetization to 
maintain the website and create high quality content while stil benefiting visitors. 


However, some pages are created to benefit the website owner or other organizations and with te to no attention paid to 
the experiance of the people who visit. When such pages use deceptive techniques to appear near the top of search 
results, it may disincentivize the creation of high quality content by crowding out pages created with time, effort, expertise 
ог original content that is helpful fr visitors. 


This section describes characteristics of spam or spam-ike pages that should be rated Lowest. In general, spam sites 
attempt to game their way to the top of search results through a variety of techniques such as repeating keywords over. 
and over, or showing search engines content thats different from what visitors to the site wil see. Hackers sometimes 
even get into legitimate sites and change them into spam sites that redirect people to scams or worse. The Google 
‘Search Central Webmaster Quality Guidelines is a good reference for non-webmasters to learn more about different spam 
techniques. If you recognize any of these spam techniques on a page, please use the Lowest rating. 


You do not need to develop spam recognition expertise or use special spam identification tools for PQ rating. Please 
review this section for guidance on what to look for even without having such tools or expertise. 
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7.5.1 Cannot Determine 





Purpose, No MC, Little MC, or Lowest Quality MC 


All pages should be created with sufficient quantity and quality of MC so that the page can achieve its purpose. If the MC 
is lacking in quality or quantity, typically a Low quality rating should be used. However, sometimes pages are so lacking 
in MC that the purpose of the page itself is unclear or you may suspect the page is not actually created for people. 


if any of the following are ruo, the page should be rated Lowest: 
* Despite your best efforts, the purpose of the page cannot truly be understood because the MC is gibberish or 
otherwise unusable for people 
* The page deliberately has no MC 
The page is deliberately created with so litte MC that it fails to achieve any purpose 
* The page is deliberately created with such low quality MC that it falls to achieve any purpose 





7.5.2 Hacked, Defaced, or Spammed Pages 


Websites can become hacked, defaced, or filled with a large amount of distracting and unhelpful content from bad actors. 
Bad actors may add unrelated MC or SC or may modify existing MC on webpages to change the purpose of the page. 
These pages should be rated Lowest because they fail to accomplish their original purpose. 





For reference, a hacked or defaced website is а site that has been modified without permission fram the website owner(s). 
Responsible webmasters should regularly check their websites for suspicious behavior and take steps to protect users. 


А common problem is "spammed" comments. Well consider a comment or forum discussion to be "spammed" if there 
are posts with unrelated comments that are not intended to help other people, but rather to advertise a product or create a 
link to a website. Frequently these comments are posted by a "bot rather than а real person. Ifa specific page on a 
website has a large amount of unrelated “spammed” comments, the page should be rated Lowest. 


7.5.3 Auto-generated MC 


Creating an abundance of content with tle time, effort or expertise with no editing or manual curation is often the defining 
attribute of spammy websites. One way to do this is to use "auto-generated" content. 


Itis possible to create many pages or even entire websites by designing a basic template from which hundreds or 
thousands of pages are created, sometimes using content from freely available sources (such as an RSS feed or API). 
These pages are created with no or very little time, effort, or expertise, and also have Ие to no editing or manual 
curation. There is litle value when pages repackage freely available content with little attention to по attention to the 
experience of people visiting the page (see hore for more information on automatically generated content). 


Itis often easiest to recognize auto-generated content by clicking on multiple pages on a website and observing 
"templated" content. It may be difficult to recognize auto-generated content from a single page, so be sure to explore the 
website if you suspect the content is templated 


Pages and websites made up of auto-generated content with no editing or manual curation, and no original content or 
value added for users, should be rated Lowest. 
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7.54 Copied MC 


Another way to create an abundance of content with litle time, effort, expertise, ес. is to copy content from another 
Source. The word "copied" refers to the practice of "scraping" content, or copying content from other non-affliated 
websites without adding any original content or value to users (see here for more information on copied or scraped 
conten) 


Pages with copied content are created with no or very litle time, effort, or expertise, and also have little to no editing or 
manual curation. There is litle value when pages are created simply by copying content from other sites. As with 
auto-generated content, it is a technique that can be used to creato many pages or even a whole website. For this 
reason, it is often easiest to recognize copied content by exploring many pages on the site. 


The Lowest rating is appropriate if ай or almost ай of the MC on the page is copied with little or no time, effort, expertise, 
manual curation, or added value for users. Such pages should be rated Lowest, even if the page assigns credit for the 
content to another source, 


Ай of the following are considered copied content: 

‘© Content copied exactly from an identifiable source. Sometimes an entire page is copied, and sometimes just 
parts of the page are copied. Sometimes multiple pages are copied and then pasted together into а single page. 
Text that has been copied exactly is usually the easiest type of copied content to identify. 

‘© Content that is copied, but changed slightly from the original. This type of copying makes it dificult to find 
the exact matching original source. Sometimes just a few words are changed, or whole sentences are changed, 
ога "find and replace" modification is made, where one word is replaced with another throughout the text. These 
types of changes are deliberately done to make it dificult to find the original source of the content. We call this 
kind of content "copied with minimal alteration.” 

* Content copied from a changing source, such as a search results page ог news feed. You often will not be 
able to find an exact matching original source if it is a copy of "dynamic" content (content that changes frequently). 
However, we wil stil consider this to be copied content. 


Important: Licensed or syndicated content should not be considered as "copied" (see here for more on web syndication). 
Examples of syndicated content in the U.S. include news articles by AP or Reuters. 


7.5.5 How to Determine if Content is Copied 


How do you determine whether all or most of the MC is copied? How do you identify the original source of the content? 
These things can be dificult to determine, but the following steps may help. 


4. Copy a sentence or phrase in the text. It may be necessary to try а few sentences or phrases from the page just 
to be sure. When deciding what sentence or phrase to сору, ку to find a sentence ог series of several words 
without punctuation, unusual characters, or suspicious words that may have replaced the original text. 

2. Search on Google by pasting the sentence or phrase (surrounded by quotation marks to search for an exact 


match) inside the Google search box. You may also try without quotation marks to search for more general 
matches. 


Google 


“Dory uses the Golden Capto summon he Wingad monkeys b 
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Here are some sentences and phrases from this page about "The Wizard of Oz" fim and how to search for 
them. 





Sentence or Phrase From a Sentence. Query to Find the Sentence or Phrase. 





Tor the query both with and without quotation marks around 

A sentence on the page: the sentence: 

[ “АП are convinced by Dorothy that the Wizard can help 

A are convinced by Dorothy that the Wizard can help them | them too" ] 

to LAN are convinced by Dorothy that the Wizard can help 
them too] 








“Try the query both with and without quotation marks around 


A phrase on the page: the sentence: 
L "Dorothy uses the Golden Cap to summon the Winged 

Dorothy uses the Golden Cap to summon the Winged Monkoys"] 

Monkeys [Dorothy uses the Golden Cap to summon the Winged 
Monkeys] 





Try the query bath with and without quotation marks around 

A diferent phrase on the page: the sentence: 

T “revealing an old man who had journeyed to Oz from 

revealing an old man who had journeyed to Oz from Omaha. | Omaha long ago in a hot air balloon" ] 

Jong ago in a hat air balloon [revealing an old man who had journeyed to Oz from. 
Omaha long ago in a hot air balloon] 














3. Compare the pages you find that match the sentence or phrase. Is most of their MC the same? If so, does one 
clearly come from a highly authoritative source that is known for original content creation (newspaper, magazine, 
‘medical foundation, еіс)? Does опе source seem to reasonably be the original? Does one source appear to 
have the earliest publication date, verified by sources lke the Wayback Machine? 


Use your best judgment. Sometimes itis clear that the content is copied from somewhere, but you cannot tell what the 
original source is. Other times the content found on the original source has changed enough that searches for sentences 
or phrases may no longer match the original source. For example, Wikipedia articles can change dramatically over time. 
Text copied from old copies may not match the current content. If you strongly suspect the page you are evaluating is not 
the original source, consider it likely to be copied. 


Based on this older version of a Wikipedia article about "The Wizard of Oz," which сап be found using the Intemet Archive 
Wayback Machine, we can confirm that the text in the table has been copied. Please note that searching for the above 
‘sentence and phrases will not lead you to a current Wikipedia article about "The Wizard of Oz,” which demonstrates how 
confirming that the MC of a page contains copied content can sometimes be difficult. 


Important: The Lowest rating is appropriate if all or almost ай of the MC оп the page is copied with litle ог no time, effort, 


expertise, manual curation, or added value for users. Such pages should be rated Lowest, even if the page assigns 
Credit for the content to another source. 
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76 


Examples of Lowest Quality Pages 








‘Stormfront, a while supremacist 
website. The site is known for 


propagating white nationalism, 
antisemitism, Holocaust denial, 
white supremacy, and 


= Promotes, condones, or incites hatred 
against a Specified Group of people 


арасат Lowest алану Characteristics [PO Rang and Exploration 
Кеш] 
maie ol | Harman реа drupe tt 


Reputation research and content on the site 
confirm that the Lowest rating is appropriate, 





This page describes a 
non-scentifcally backed theory 
© superiority of a Specified 
Group. 


‘+ Promotes, condones, or incites hatred 
against a Specified Group of people 

‘+ Contains unsubstantiated theareslaims 
not grounded in any reasonable facts ог 
evidence 


Islamophobia, 
Harmful to Specified Groups, Harmfully teat 
Кети] Misleading Information Lor Melun Het 


The purpose of the page is to promote the 
мен of superiority of a group of people. 


This page makes unsubstantiated claims 
bout the role of melanin in the human body. 








This post encourages mixing ап 
antianxiety drug with a 
recreational drug 


Harmful to Self or Other Individuals 


=. Encourages, depicts, incites, or directly 
causes physical, mantal, emotional, or 
financia harm to self or others 


ا ل 
ime Nedim Hem‏ 


This post and other content on tho page 
encourages dangerous behavior that could 
cause ham. 





Lowest: Ealing disorder 
encouragement (YMYL) 


This pago has posts that 
encourage people to lose weight 
in unhealthy ways. 


Harmful to Self or Other individu 
Harmfully Misleading Information 





Encourages, depicts, incites, or directly 
causes physical, mental, emotional or 
financial harm t self or others 

^ Contains information tnat contradicts 
well-established expert consensus 


Berapa pa pel 
ime! Medium Hem 


‘This post encourages harmful behavior. 


"Proana" or pro-ancrexiais a movement to 
view anorexia ав а lifestyle choice, not a 
disease, Proana and similar "hinspratn" 
Pages contradict wal-estabished expert 
Consensus hal anorexia an eating disorder. 





шиза: Personal information 


‘lection seam (YMYL) 





Superfclally, this looks ike а 
payment page on a merchant 
sie. 


However, the form asks for highly 
Sensitive personal information 
Such as social security number, 
divers license number. Бага 
account numbers, and ATM PIN. 








Harmful to Self or Other Individuals, 
Untrustworthy 


© Pages that have characteristics of 
scams, malicious downloads, or other 
armil behavior 

© Encourages, depicts, incites, or direct 
causes physical, mental, emotional or 
financial harm t self or others 





White this оска ike а merchant site, the true. 
purpose ofthis page is to collect information 
{hat could be used la commit identity thet or 
remove money from someone's bank account. 
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[Wabpage/Content Lowest Quality Characteristics. [Pa Rating and Explanation. 
Harmful to Self or Other Individuals, هيمد‎ 
Untrustworthy Lees Medium Het 


and possible scam (YMYL) 


Tris в an article suggesting 
licking on a link to avoid paying 
debt 


© Lowest reputation or lowest E-A-T 

Pages that have characteristics of 
scams, malicious downloads, or other 
armil behavior 


* Encourages, depicts, incites, or direct 


causes physical, mental, emotional, or 
financial harm t self or others 


This page has multiple conceming 
characteristics. The page has lowest E-A-T as 
it clearly lacks the financial expertise 
necessary о give financial guidance: licking 
оп tha link could lead to a potentially harmfut 
outcome. Overall the promise of debt relief is 
э соттоп scam осте, 





Lowest: Miley Cyrus is dead 
Г 2 

asso 

This is a factually inaccurate 
article on a website called "Now 
E News”. It falsely claims Miley 
Cyrus is dead. 


Harmfully Misleading Information, 
Untrustworthy 


* ^ Contains сапу inaccurate information 
"hat can easily be refuted by 
straightforward and widely accepted 
facts 


Deceptive design or deceptive intent 


This website is designed to look ike а news 
site and yet has deliberate factually inaccurate 
information. There is по information on who is 


responsible for ihis content. This page was 
designed to misinform or deceive. 





potential scam (YMYL) 


This article explains how crystals 
can be used 10 help cure 
cancers. It has a prominent 
“Shop Crystals" link at the top. 


Harmful to Self or Other Individual 
Harmfully Misleading Information, 
Untrustworthy 


Contains unsubstantiated thecras/laims 
not grounded in any reasonable facts ог 
evidence 

+ Contains information that contradicts 
wal-astablished expert consensus 

Pages that have characteristics of 
scams, malicious downloads, or other 
harmful behavior 

© Encourages, depicts, incites, or directly 
causes physical, mental, emotional or 
financial harm to self or others 


| 
мешт Honest 


The claims made about the healing power of 
crystals contradict well-established expert 
sciens and medical consensus. 


Seling "oures" for diseases such as cancer is 
а commonly used scam technique. 








Lowest: Flat wath (YMYL) 


This в a homepage of The Fiat 
Earth Society, a group o people 
dedicated to spreading the belief 
that be earth ls lat 





Harmfully Misleading Information 


= Contains unsubstantiated theoras/laims 
not grounded in any reasonable facts or 
evidence 

< Contains clearly inaccurate information 
"hat can easily be refuted by 
straightforward and widely accepted 
facis 

^^ Contains information that contradicts 
well-established expert consensus 








The purpose of this website is promoting 
misleading information. It is harmful to people 
and society as it undermines trust in public 
institutions. 
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[Wabpage/Content Lowest Quality Characteristics. [Pa Rating and Explanation. 
ية‎ 
Lees Nedim He 
Lm Untrustworthy There is no information about who created this 


This YMYL medical site has 
мега no information about who 
is responsible for the content or 
the website. 


» inadequate information abaut the 
website or creator of tne MC for the 


purpose of ho page 


website, no contact information, and по 
Information about the autho, 


Medical pages requie a high degree of user 
trust. Because here is по information about 
Who owns is website and who created this 
content this is considered an untrustworthy 
website. 





Lowest: Merchant wih it 





characteristies (YMYL) 


This looks ке a product isting 
Page on a merchant website, 


Untrustworthy 


‘Inadequate information about the 
website or creator af the MC forthe 
purpose of the page 

+ Characteristics of scams or other harmful 
behavior 


iMi ed 
Levee! Melun Het 


inaccurate information about shipping and 
retums, deceptive use of logos and no 
Information about who is responsible make this 
shopping sile appear potentially fraudulent and 
untrustworthy. 


This page is seling Nike Air Jordan shoes. 
However, there is inadequate contact 
information on the contactus page for a 
merchant ste 


in addition, the Shinning and Retuma page 
has tha name of another company that seems 
to be unrelated. There are also official ooking 
logos at the bottom of the homepage, including 
the Better Business Bureau logo and Google 
Checkout logo, that don't appear lo be 
afliated with the website 








Lowest: Stomach fu lowest EAT 
Aricle (YMYL) 


This poorly writen article 
describes how to cure he 
‘Stomach u 


Harmfully Misleading Information, 
Untrustworthy 


+ Contains information that contradicts 
Wal-astablished expert consensus 

= Inadequate information about the 
website or creator of the MC for the 
purpose of ho page 

© Lowest reputation or lowest E-A-T 


mapa pea pad 
Hanes 


The writing is poor and the information is not 
consistent with standard medical guidance. I 
is clear from the content the author does nat 
have medical expertise. 


This page should not be rusted as a medical 





Lowest: Pim 
EAT айк (YMYL) 


lowest 


This very poorly writen article 
describes popping pimples. 





Untrustworthy 


Inadequate information about the 
website or creator ofthe MC for the 
purpose of the page 

© Lowest reputation or lowest E-A-T 





tae 
Lees Nedim Het 


The writing is extremely poor and thera is 
virtually no information in this article. 


For example, the opening sentence is 
"Popping pimples could be or could be not the 
пен trend of geting rid of hem." 


‘tis clear from the content that he author does 
not have skin care expertise. 
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[Wabpage/Content 


Lowest Quality Characteristics 


[Pa Rating and Explanation. 





Loussi: Malicious merchant siie 
[n 


This website appears io be an 
eyeglasses merchant ste 


Harmful to Self or Other individual 
Untrustworthy 





= Lowest reputation or lowest E-A-T 

Characteristics of scams or other harmful 
behavior 

© Encourages, depicts, incites, or directly 
causes physical, mental, emotional, or 
financial harm t self or others 


gea papa 
Honest 





This website swindled people and physically 
threatened people wha complained, These 
articles on Wikipedia and the New York Times 
describe he deceptive techniques used by this 
Website and provida char negative information 
‘bout the website and its owner. 


‘The reputation of this website is lowest based 
оп reports of malicious behavior. 





Lowest: Chicken recipes no MC. 


This looks like а list of chicken 
recipes, but is actually a list of 
links that are ade. 


Untrustworthy, Spammy 


= ммс 

© Content created with itl to no time, 
олоп, expertise, manual curation, or 
added value for users. 

© Deceptive design or deceptive intent 


ate m 
Lost Nedim Het 


MC should be created to benefit people who 
come lo the website. Pages without MC 
should bo considered spammy as they crowd 
Ош and disincentivize the creation of pages 
чап high quality MC. 


Pages with Ads that are designed to look ike 
МО should be considered untrustworthy. 





Lowest: Mesothelioma deceptive 
recta (YMYL) 


This page looks ike a isting of 
resources about Mesothelioma, a 
form of lung cancer. 


Untrustworthy, Spammy 


e Nome 

© Content created with itl to no time, 
effort, expertise, manual curation, or 
added value for users. 

© Deceptive design or deceptive intant 


يومد 
Loret Nedim Honest‏ 


This page is designed to look like a directory 
page or a list of helpful resources, but every 
link ls an Ad. There is по MC on this page. 


Pages without MC should be considered 
spammy as thay crowd out and disincentivize 
tha creation of pages with high quality MC. 


Pages that disguise Ads as directory listings oF 
oiher MC should be considered untrustworthy. 








Lowest: Ads deliberately obscure 
Mc 


When you fst visit the page, t 
appears that there is no MC, опу 
As. It you scroll down, you wil 
‘eventual find the MC at he 
bottom. 





Untrustworthy 


6 Dolieratoly obstructed or obscured MC 








This page requires scroling past so many Ads 
to getto the MC that many people may stop. 
scroling and leave the page before reaching 
the MC. 


Deliberately obscured МС to benefit the 
website with no help for people visiting the 
Page should be considered untrustworthy. 
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[Wabpage/Content Lowest Quality Characteristics [Pa Rating and Explanation. 
وة‎ 
Lees Nedim о Her 


Lowest Deceptive and 
untrustworthy fake survey page 


This page appears to be a 
survey with the opportunity to win 
ап Apple MacBook Air, an 
iPhone Sc, or a Galaxy S4 


Untrustworthy, Spammy 


© Deceptive design or deceptive intent 

® Pages that have characteristics of 
cams, malicious downloads. or other 
harmful behavior 

© Inadequate information about the 
website or creator of tne MC for the 
purpose of the page 

Content created with litle to no time, 
‘fort, expertise, manual curation, or 
эдей value for users 


This page contains а fake survey, which 
appears to promise users the opportunity to 
win an Apple MacBook Ar, an IPhone Se, or a 
Galaxy 54. 


However there is no information about who is 
responsible for the survey, even though upon 
clicking, the survey asks for highly personal 
Information such as bank account numbers. 
The message atthe top ofthe page, “Thank 
You from Goggle com and survey content 
саша mislead users to think that the survey is 
lated wiin Googie. 


This pago is deceptive and shouid be 
considered untrustworthy. 





Lowest: Lowest MC free credit 
moan (YMYL) 


The tle of the page suggests 
that t may provide а credit raport 
from the government 
"Yreacredivepest gov” 


However, the page has gibberish 
МС and spammy looking 
features, such as ads or links 
surrounding the meaningless 
ps 


Untrustworthy, Spammy 


* ^ Deceptive design or deceptive intent 

® Pages that have characteristics of 
scams, malicious downloads, or other 
harmful behavior. 

* Inadequate information about the 
website or creator of the MC fr the 
purpose of the page 

+ Cannot determine any purpose, e.g. 
gibberish MC 

+ Content created with litle to no time, 
effort, expertise, manual curation, ог 
added value for users 


anaana ansa] 
мешт eger 


The MC is gibberish. There is no beneficial 
purpose of gibberish МС. The tite of the page 
is misleading, given the MC. 


This pago also has deceptive features in the 
supplementary content, such as a friend 
request, а prize alert, and a download button. 
"tis highly likely that these are ads or 
malicious downoad inks. 








Lowest: Deliberately created wth 
mme 


This в a confusing page. It 
seams to be missing MC and 
instead has a block of ade. It 
тау be that the pago was 
created to have content about a 
video game called Point Blank. 


The only content related to Point 
Blank is a video from another 
website embedded in tne right 
hand side of he page. 





Untrustworthy, Spammy 


«Cannot determine any purpose, e.g. 
gibberish MC 

6 No MC or insufficient MC for the purpose 
of the page 

= Content created with litle to no time, 
effort, expertise, manual curation, of 
added value for users 

= No information about the website or 
creator ofthe MC 





ام 
Nesim O Mene‏ 


Who this page may look odd or broken, many 
other pages on the website are similar pages 
жан oniy ads. 


Many don't have videos, and any videos 
present are trom another website, not original 
fo this one. 


This page and this website was created with 
lila time and adds no value for users. 
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[Wabpage/Content 


Lowest Quality Characteristics 


[Pa Rating and Explanation. 





Lowest: Buy Xanax 


This в a page on the website for 
же U.S. Green Building Counci! 
However. this page has the 
words “Buy Xanax" over and 
over again, along with related 
words about buying Xanax and а 
URL lik 


Untrustworthy, Spammy 


^^ Unmaintained websites, and hacked, 
defaced, or spammed pages 

Pages that have characteristics of 
scams, malicious downloads, or other 
harmful behavior. 


gea papa 
Honest 





This specife page has been created or 
modified on this organization's website in 
hopes that people wil cick оп the link to buy 
tha prescription drug Xanax 


This is а spam technique. Spammers hope 
that the page wil rank well in search results 
because te website is otherwise high quality. 
Tho words "buy xanax" are used repeatedly in 
hopes tat the page wil show up for the 
search "buy xanax 


Attempting to врат search engines and tick 
people into clicking on inks is highly 
untrustworthy bahaviar. 








endis code (YMYL) 


This page is full of 
supplementary content, inks, 
and large ads. 


The MC at the top gives 
Information about a specific error 
code P2570 and a second block 
of main content inthe middle 
‘shows more information about 
P2570. 


The MC is broken up by large 
ads and iie dificult to 
understand due to the poor 
content quality. 





Untrustworthy, Spammy 


Content created with itl to no time, 
effort, expertise, manual curation, or 
added value for users 

^ insufficient MC for the purpose of tha 
page 

* Inadequate information about the 
website or creator of the MC for the 
purpose of he page 





mese us ren] 
Lees Medium He 


This page has low quality MC and spammy 
atributos 


Tho page repeals the error code "P2570" 
beyond what в helpful for users- a technique. 
known as irrelevant keywords 


The small amount of MC is not original to the 
website. Lists of error codes are available 
either by purchase or by scraping 


An important actis missing at the top “When 
you check engine ight сате on code P2570 
tha reason should be "The reason is left out - 


this should bo the most important content on 
"he page! 


Looking at other pages on the website, itis 
clear that each page is created automaticaly 
by fling in a template trom a ist of errar code 
information. There appears lo be litle to no 
manual ediing or checking to make sure the 
eror code contant is correctly pulled in, as 
evident by the missing reason on this page. 





As with many spammy websites, there is no 
information about who is responsible for this 
website and no way ta report problems with 
the page 
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[Wabpage/Content Lowest Quality Characteristics [Pa Rating and Explanation. 
تيميد‎ 
Lort Nedim Her 


werde wipes (YMYL) 


This page claims to ist the best 
bulldog wrinkle wipes and uses 
fate inks. 


Untrustworthy, Spammy 


© Deceptive design or deceptive intent 

© No beneficial purpose 

Content created with tle to no time, 
fort, expertise, manual curation, or 
added value for users 

Inadequate information about the 
website or creator af the MC for the 
purpose of ha page 


The MC on this page is copied with itie or no 
"ime. efor, expertise, manual curation, or 
added value for users. 


Looking at the content at the bottom of the 
page and exploring other pages on the 
Website. itis clear that each page в created 
automatically using content от Amazon and 
Substituting diferent product names into а 
Generic template. There appears to be ile to 
по manual editing or curation on this page. 


The true purpose o this page is to bring users 
seeking information about products to this 
website in hopes that they wil cick and order 
products by clicking on Amazon affilala inks. 
This website does not have a beneficial 
purpose for users but rather was created to 
Benefit the creator of the website. Though the 
website discloses that thay use aflat links, 
the site offers no added value for users. 


As with many deceptive websites there is no 
information about who is responsible fortis 
Кз 











Lowest Viagra copied cont 
[m 


This looks ке an information. 
website for tha drug Viagra. 





Untrustworthy, Spammy 


= Deceptive design or deceptive intent 

No beneficial purpose 

Content created wilh tle to no time, 
effort, expertise, manual curation, of 
added value for users 

Inadequate information about the 
Website or creator of the MC Tor the 
purpose of he page 





AM MC on this page is copied with ite or no 
"ime, efor, expertisa, manual curation, or 
added value for users. Here isthe оша 


Exploration of the pagas on this website are 
similarly created with ile expertise or value 
added for users. 


The true purpose o this page is to bring users 
seeking information about Viagra to this 
‘website in hopes that they wil cick on he 
“order now” link. The “order now link goes to 
another website and is likely monetized. 
Therefore, inis website does not have a 
beneficial purpose for users but rather was 
created to benefit another website. 


As with many deceptive websites, there is no 
information about who is responsible for this 
нараа. 
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8.0 Medium Quality Pages 


In this section, we will describe pages that should get the Medium quality rating. Medium pages have a beneficial 
purpose and achieve their purpose. 


There are two types of Medium quality pages: 





Discussion 








The page achieves its purpose, however, it does not merit a High quality rating, пог is there 
anything to indicate that a Low quality rang is appropriate 





‘Mixed with strong High quality 
rating characteristics 


The page or website has strong High quali rating characteristics, but also has mid Low quality 
characteristics. The strong High quality aspects make it dificult to rate the page Low. 





8.1 — Examples of Medium Quality Pages 








Medium: Encyclopedia 


Wikipedia article about baroque 
pears 


= Nothing wrong, but nothing special 


Wobpage/Type of Content — Medium Quality Characteristics PQ Rating and Explanation 
ктен 
Loves Mesum Highest 


“This is a short Wikipedia arie about baroque 
ears, а fairy narrow topic. This page is OK 
for its purpose, but it doesn't display 
haraclenetcs associated with a High rating. 





Medium: Humor 


Pago from a humorous sile 


M Nothing wrong, but nothing special 


кетен 
Loves Mesum Hes 


This page is from a humorous site that 
‘encourages users to post photos with mouths 
drawn on them. This paga is OK for ts 
purpose, but it doesn't display characteristics 
sociated with a High rating. 





Medium: Entertainment 1 


Aride about ‘Keeping Up with the 
Kardashians" show 


= Nothing wrong, but nothing special 


merece 
Loves Mesum шш. 


This page в оп a website dedicated to 
entertainment news. This page is OK for ls 
Purpose, but it doesn't display characteristics 
associaed with a High rating. 





Medium: Entertainment 2 


Алде about Miley Cyrus 








M Nothing wrong, but nothing special 





moment 
Loves Mesum Highest 


This page is from a newsientertainment 
website. This page is OK for ts purpose, but it 
doesn't display characteristics associated with 
a High rating 
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Wobpage/Type of Content — Medium Quality Characteristics PQ Rating and Explanation 
корш peter 
Median: Q&A. Lowest Ven Het 


Q&A page where a user is looking 
for advice on where to buy high 
quality women's clothing online 


= Nothing wrong, but nothing special 


This ОВА page has a discussion of diferent 
alin merchants. Thera is some everyday 
expertise, but t doesnt display characteristics 
associated with a High rating. 





Medium: Page with Erro 
Message 


"Custom 404° page 


* ^ Nothing wrong, but nothing special 


prepa pem 
Lowes Medium Honest 


This в an example of a "custom 404” page. 
“These pages are designed to alert users that 
the URL they аге trying to visit no longer exist 
‘Some websites do а nice job of not only 
alerting users about a problem, but also giving 
them help. 


“This page is on a welLknown merchant website 
with a good reputation. However, this particular 
раде displays the bara minimum of content 
headed t explain the problem to users, and. 
the опу help offered is a link to the homepage. 





мейш. Informational 


Page about propulsion on the 
“Quality Reasoning Group” 
section ofa university's website 


о Mixed, but has some EAT 

© Website has a good reputation. 

* Contant is оу to be accurate and 
trustworthy because of the website 
though no reference links are provided 


=i 


Although this is a wel-known, highly respected 
‘university with а high quality site, this page is 
оп а vary specialized section of the university 
‘Website. No author is isted and the page may 
have been a one-time project, possibly rom a 
student, which is по longer maintained. 





Median: Lyrics 


Song ries for the song “Never 
Youfrear Love" 


M Nothing wrong, but nothing special 


fares one] 


loves Vedum Highest 


‘There are many lyrics websites that have 
similar content. This page is OK for its 
Purpose, but it doesn't display characteristics 
associated with a High rating. 








Madum: Recipe 1 


Mexi-Chicken Casserole on a 
newspaper website 





= Nothing wrong, but nothing special 





jamjam 


Lors Medium Highest 


This recipe was contributed by an author of 
cookbooks. However, the page has no SC 

od to the purpose of the page, such as 
views or inks to other recipes, etc. This page 
% ОК lor its purpose, bul it doesn’ display 
characteristics associated with a High rating. 
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Wobpage/Type of Content 


‘Medium Quality Characteristics 


PQ Rating and Explanation 





Medium: Reine 2 
Recipe for cherry-topped cake 


* ^ Nothing wrong, but nothing special 


This websia is known for high quality content 
about animals and the environment. This 

articular pago has a recipe for kids. There 
n't much MC or SC. This page is OK for its 
Purpose, but it doesn't display characteristics 
associated with a High rating. 








Medium: Video 


Video of a kitten meowing a lot 


* ^ Nothing wrong, but nothing special 


= 


Medium о нуш 


‘This is а professanal-created video of a cule 
kiten meowing. This page is OK fr its 
Purpose, ut i doesn't display characteristics 
associated with a High rating 





Medium: Forum 1 


Forum page on an onine auction 
website. A user is looking for 
help choosing a product category. 


M Mixed, but with some redeeming qualities 


mired 


The relative lack of MC is balanced a bit by the 
‘expertise of this forum. This forum is dedicated 
to his kind of question. 








Mediam: Forum 2 


Forum page on a dance website. 
A usar is looking for advice on 
how to wash ballet shoes. 


^ Mixed, but with some redeeming qualities 








pomo mis is 


Love мет Highest 


This forum is about dance topics, and many 
pages have expertise кот a community of 
ballet dancers. On this particuiar page, 
participants have everyday experiance washing 
ballet shoes and make recommendations 
based on ther oun experiences. 


This page is mixed" because there is some 
distracting content that makes it hard lo read 
the MC. However, this в not a beauty contest 
Even though the page may be cluttered, there 
is some valuable everyday expertise and 

хар МС, making Medium a good rating for 
this page. 
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9.0 Page Quality Rating Tasks 


At frst, РО rating may seem difficult. There are several aspects of the page and the website to look at and think about. 
This type of rating takes practice. Rereading sections of these guidelines and thinking about the examples may help 
when you encounter dificul rating tasks. 


Important: Do not struggle with each РО rating. Please give your best rating and move on. If you are having trouble 
deciding between two ratings, use the lower rating. If you are torn between three ratings, choose the one in the middle. 


Do not consider the country or location of the page or website for РО rating. For example, English (US) raters should use 
the same PQ standards when rating pages from other English language websites (UK websites, Canadian websites, etc.) 
as they use when rating pages from U.S. websites. In other words, English (US) raters should not lower the РО rating 
because the page location (UK, Canada) does not match the task location 


These guidelines are specific to “regular” webpages. Occasionally, you may be asked to rate а landing page that is not a 
webpage. Far example, you may be asked to rate a PDF file, a PNG or JPEG image fle, etc. When the landing page of 
the URL is not a webpage, some of the criteria in these guidelines may not apply. In this case, please use your judgment. 





Finally, this Page Quality Rating Guideline does not completely cover every aspect of page quality. If you find pages that 
you truly believe to be High or Low quality. please rate them as such, even if the reason Is based on something not 
‘covered in this document. Please use the comment section to explain your reasoning. As always, use your judgment. 


9.1 — Instructions for Rating Page Quality Tasks 
The Page Quality task page is broken up into several parts: 

‘Some intial questions about the task landing page. 

А PQ grid" to record your observations about PQ characteristics of the landing page. 


‘The Overall PQ rating slider which records your Overall PQ rating, 
А comment box to explain your rating. 


‘Some results to the initial questions will end the task early. Ifthe page is Porn, Foreign Language, Did Not Load, ог 
Consists of restricted or inaccessible МС (e.g. subscription is required to view MC), you will not fil in the PQ grid or assign. 
an overall rating 


* Foreign Language should be used when the language on the landing page is NOT the task language, a language 
that is commonly used by a significant percentage of the population in the task location, or English. 

* Did Not Load should be used for pages where there is absolutely no content on the page created by the website. 
There is no MC, SC, or Ads on the page. See this Wikipedia article for descriptions of different types of error 
messages. 


Similar, if you respond that the page is malicious, harmful, deceptive, or lacking in purpose, you will also not fll in the РО 
grid or assign an overall rating. 


The РО grid is designed to be your "note pad.” It allows you to record your observations about the landing page and the 
website it belongs to. 
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3.1.1 Rating on Your Phone 


You should open the task landing page on your phone using the standard Send to Device feature, unless otherwise. 
instructed. However, more intensive analysis on the website can be done on your computer. For example, you should 
‘open and explore the links that are provided to help you do research on website information and reputation on your 
computer. 


Important: Clicking on the task URL may bring up an interstitial page. You can ignore this page in your rating criteria i 
you can easily get to the MC. However, ifthe interstitial page makes it extremely hard (or impossible) to get to the MC, 
that should factor into your Page Quality rating. 


9.2 — Reputation and E-A-T: Website or the Creators of the Main Content? 


You must consider the reputation and E-A-T of both the website and the creators of the MC in order to assign a Page 
Quality rating. 


‘The reputation and E-A-T of the creators of the MC is extremely important when a website has different authors or content 
creators on different pages. This is true of forum and Q&A pages, news websites, or websites that have user-generated 
content, such as YouTube, Twitter, etc. The reputation and E-A-T assessment for pages on these types of websites may 
differ drastically depending on what page you are evaluating. There are Highest quality YouTube videos created by 
highly reputable and expert content creators, as well as Lowest quality YouTube videos created with a dangerous lack of 
Е.А on YMYL topics. 





Important: Research the reputation and E-A-T of both the website and the creators of the MC. If either are lacking for 
the purpose of the page, the Lew or Lowast rating is appropriate, 
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10.0 Page Quality Criteria for Spe: 





ic Types of Pages 


10.1 Ratings for Encyclopedia Pages 


There are many encyclopedia-lype websites. Some are highly-respected publications that are standard references, while 
some are websites with content created and edited by anonymous users, with no editorial oversight or fact checking. We 
may not always know the author of the specific encyclopedia article, and therefore must rely on website reputation 
research to determine the E-A-T of the article. High and Highest quality ratings should only be used for encyclopedias 
with very good reputations for accuracy and expertise, where the article itself is well-researched with appropriate 
references cited. 


A note about Wikipedia: in general, the website has a good reputation and is a very popular resource that is generally 
valued for accuracy. However, there is no single author or organization that vouches for the accuracy of Wikipedia 
articles, and the quality of pages varies. You should perform page-level checks on individual articles. 





A Wikipedia article on а non-YMYL topic (example) with a satisfying amount of accurate information and trustworthy 
‘external references can usually be rated in the High range. Some Wikipedia articles may even be rated higher, although 
remember that a high level of expertise is required for YMYL topics. А РО rating in the Medium range is often appropriate 
for pages with less extensive MC and extemal references. Naturally, Wikipedia articles with very ltle MC should get 
lower РО ratings. Factual inaccuracy is а sufficient reason for a Low or Lowest rating. 


10.2 Ratings for Pages with Error Messages or No MC 
‘Some pages are temporarily broken pages on otherwise functioning websites, while some pages have an explicit eror (or 
custom 404) message. In some cases, pages are missing MC as well. Please think about whether the page offers help 
for users—did the webmaster spend time, effort, and care on the page? 


Here are some examples. 








Wobpage/Type of Content Characteristics of the Page PQ Rating and Explanation 
او‎ 
Lovet MeXum Highest 


This is an example of a page with no MC. You 
might think that һө MC а "missing dua toa 
problem wit this particular page, but in fact, 
‘his website has hundreds of pages that оок 
the same—no MC, just Ads. This website 
‘shows Ads with itie or no attempt to help 
users, and should be rated Lowest quality. 


Lwes Майит онды 


= Large amount of prominent Ads and no 


iberalely crested with no 
————————— attempt lo help users 





= No MC (probably due to a temporary 
age didn't uly load and has no technical err) ‘This page has no MC and no eror message. It 
мс +  Sificen SC is an isolated example of a page with no MC or 
‘© Posiive reputation error message on a website for a reputable 
‘newspaper for a town in Michigan. Al of the 
navigation Inks work, and the page was later 
С 
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Webpage/Type of Content 


Characteristics of the Page 


PQ Rating and Explanation 





message i 


= Nothing wrong, but nothing special 


[pee reae 
Loves Mesum | Hone 


This в an example of a "custom 404” page, 
alerting users that the URL thay are tying lo 
isit no longer exists. Some websites do а nice 
ob of alaring users about а problem and 
Providing helpful tps. 


This page is on a wellknown merchant website 
with а good reputation. However, this particular 
Page displays the bara minimum of content 
headed о explain the problem to users, and 
the only helpful contentis а ink to the 
homepage 





= A satisfying amount of high quality MC for 
its purpose 

+ Help SC that improves the user 
experiance 

+ Posiive reputation 


—+ 


= 1 
‘This в an example of a custom 404" page, 
alerting users that lhe URL thay are tying fo 
isit no longer exits. This website does a nice, 
Job of explaining the issue and providing helpful 
fips. including a search box. 





message 3 








© A satisfying amount af high quality MC for 
its purpose 

^ Нер SC that improves the user 
experiance 

e Very positive reputation 





n 


= 


‘This в an example of a custom 404" page. 
These pages are designed to alert users that 
the URL they are trying to visit no longer exists. 
The MC of this page is the cartoon, the caption, 
and the search functionality, which is specific 1o 
‘the content of the website. Il is clear that tme, 
‘fort, and talent was involved in the creation of 
the ic. 


This publication has a very positive reputation 
and is spacficaly known for its cartoons, which 
‘allows us to go as high as Ноћ» to Highest 





10.3 Ratings for Forums and Q&A pages 


Ratings for forum and Q&A pages can be challenging. The most important aspect is the E-A-T of the participants in the 
discussion, which can be dificult to judge. Keep in mind the following: 


* The Main Content on forum and Q&A pages includes both the question as well as the answersiresponses and 


resulting discussions. 


‘© Rate forum апа Q&A pages from the point of view of a user who visits the page, rather than a participant involved 


in the discussion. 
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Неге are some examples. 











^ YMYL page with inaccurate potentially 
dangerous medical advice 


Webpage/Type of Content — | Characteristics of the Page РО Rating and Explanation 
кезең 
мы Wen ne 


We must evaluate this page from the point of 
View of a user visiting this раде from a search 
engine, rather than a participant. Tha question. 
i poory worded and dificult to understand. 
The answers ara poorly worded and have 
incorrect and potentially dangerous medical 
advice, making it lowest quality MC. 





G&A page with unanswer 


= Misleading page design 
5 Unsaistying amount of MC for the 
Purpose ol the page 


pf 
мешт ges 





In addition to having no answer, this page has 
‘Ads and links to other questions (misleading 
labeled as "Relevant answers") displayed 
prominently, which users may mistake for 
answers to ihe question. It takes a moment to 
lica that this page actually has no answer. 
Deceptive design and lack of an answer make 
this page а frustratingly poor user experience 
‘and cause this page lo completly fail lo 
achieve its purpose. 





a (MYL) 


‘+ Misleading page design 
(© Unsalishying amount of MC for the 
purpose of Ine page 


н 


jp 


Medium Highest 





In this example, the MC is boxed in red. 
Please read the MC, including he completely 
unhelpful “answer to the question in the red 
box. This answer is so unhelpful, we can 
consider this question to be unanswered. This 
Page has an unsatisfying amount of MC. 


In addition to a very unhelpful “answer the 
page design makes it difficut to distinguish the 
МС from Ads. For example, below tha answer, 
же see a "sponsorad answer” which has the 
Same format ав the real answer bul is actually 
тэп Ad and not an answer ta the question. This 
pago design is somewhat misleading 











+ Unsatisfying amount of MC for the 
purpose of the page 





pepeg 
Lores emm 


‘Some websites rely on users to create virtual 
all of their MC. In this case, the MC is the 
User's question. If there ae no answers, tha 
amount of MC on the page is unsatisfying. 
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Wobpage/Type of Content | Characteristics of the Page PQ Rating and Explanation 
sacral ena 
мш Meme teen 


customs (YMYL) 


Lacking E-A-T for the purpose of the 
page 


There aro 94 answers to this question with a 
few results that seem helpful. Many of he 
posts are wrong or misleading. including the 
{op answer, which is labeled the "bast answer” 





Eorum page on how to we 


Mixed, but with some redeeming qualites 


—— 


= 


мет о нуш 


та forum is about dance topics, and many 
pages have expertise rom a community of 
balet dancers. On this particuiar page, 
participants have everyday experience washing 
ballet shoes and make recommendations 
based on ther oun experiences. 


“This page is mixed" because there is some. 
distracting content that makes it hard lo read 
the MC. However, his is nat a beauty contest! 
Even though the page may ba cluttered. here 
is some valuable everyday expertise and 
helpful MC, making Medium a good rating far 
this page. 





Everyday expertise forthe purpose of the 
page 


к==р} 





‘Many participants share their personal 
experiences with these products, giving details 
‘Such as how well certain models work with pet 
hair. There are many descriptions of 


participants! oum experiences with this product 
End how well works far tham. 





G&A vage about interviewing at 
ашал Bear (YMYL) 


Everyday expertise for the purpose of the 
page 





а 


Lost Mesum ишш. 


The answer on this page is written by an 
employee of Build-A-Bear, so it affars a 
Somewhat unique and presumably somewhat 
‘expert answer. 


“The page design alows users to read the MC 
as the Ads are clearly labeled, as well as boh 
the question and answer. 








technique in kritina 





High level of E-A-T for the topie and 
purpose of the page 





=i i 


The person who posted the frst message on 
is forum page provides a helpful resource on 
how ta mastar the magic loop technique in 
‘ting. She also shares picturas of har own 
version using an old pair of bue jeans. With 
aver 20 years of experience kniting socks, wo 
would consider har to be an expert on the topic. 
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Wabpage/Type of Content 


Characteristics of the Page 


PQ Rating and Explanation 





ancer patents Ive (YMYL) 


© Everyday expertise 
$ Asaistyng amount of high quality MC 


= 


The question on tha page asks how long 
people live with cancer. There are many 
fusis describing пон ong a loved one lived 
after diagnosis. There is very litle medical 
‘advice and the focus of the page is sharing 
personal experience, Many results аге heartfelt 
and wel writen. 





+ High level of E-A-T for the topic and 


purpose of the page 
© А satisfying amount af high quality MC 


—+ 





= 
This forum is well-known for discussions on 
Juxury designer purses. On this particular 
forum paga, members are consulting forum 
experts who have expertise authenticating bags 
from this brand. These experts can talif a 
particular bag is authentic or fake. While there 
"s ап ad at he top and a few ads within the 
forum message, it does not distract from the 
MC. which is easy to fnd. 





^ High level of E-A-T tor the topic and 
purpose ol the page 
= А satistying amount af high qualty MC 


spss 


= 


“This forum is well-known tor its community of 
‘experts on quality assurance. The responses 
эе authortative and trustworthy for the 

Purpose af the page. While there are Ads on 
‘the page, itis easy to find the МС and they are 
eariy labeled as "Sponsored Link" 








Forum page on landscaping an 
ашатат 


^ High level of E-A-T tor the topic and 
purpose of the page 
© А satisfying amount of high quality МС 


pompa 


= 1 


This discussion focuses on landscaping for a 
particuiar paludarium (an aquarium wih 
terrestrial and aquatic elements). There is a lot 
of discussion and interaction between forum. 
‘members about the types of materials and 
Species used in the aquarium. The posts show 
‘expertise in the riche topic of aquarum 
landscaping 











= Very high level of E-A-T for tho purpose 
ofthe pago. 

< Магу positive reputation, and the website 
is responsible for all content on the 
Website 





prm pamm 


E 


The purpose of the page is to answer questions 
about the usage of abbreviations. This website 
has a very good reputation as a reference for 
information on writing, publishing, ete. Itis 
‘considered highly authoritative and trustworthy 
for the topic af the page. 
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110 Page Quality Rating FAQs 





‘Question 


Answer 





Why do we have to do all these 
steps? This takes a long time! 


With practice, the amount of time needed for accurate PQ ratings wil decrease. The 
steps are important and are designed to help you assess many diferent aspects ог 
РО. You may be surprised by what you find. Pages that initially look Low quality 
may lum out to be Medium or High quality with careful inspection. The reverse may 
happen as well. We want your most informed, thoughtful opinion. 





‘Aro we just giving High quality 
ratings to pages that "look" 
‘good? 


No! The goal is to do the exact opposite. These steps are designed to help you 
analyze the page without using a superficial “does it look good?” approach. 





You talked about expertise 
when rating MC. Does 
expertise matter for all topics? 
Aren't there some topics for 
which there are no experts? 


Remember that we are not just talking about formal expertise. High quality pages 
involve time, effort, expertise, and talent/skil. Sharing personal experience is a form 
of everyday expertise. 


Specifically for content creators, everyday expertise can be assessed based on the 
talentskil level depicted in the MC (e.o., great hairstyling advice, painting/crafng 
abilities, skilful home/DIY work, etc.). In cases where the content creator is not 
‘demonstrating formal or everyday expertise but is not doing any harm, Medium is an 
appropriate rating. 


Pretty much any topic has some form of expert, but E- 
YMYL pages. 





Tis especially important for 


For most page purposes and topics, you can find experts even when the field itself is 
niche or non-mainstream. For example, there are expert alternative medicine 
websites with leading practitioners of acupuncture, herbal therapies, etc. There are 
also pages about altemative medicine written by people with no expertise or 
experience. E-A-T should distinguish between these two scenarios. 


‘One final note: if the page is harmful to people or society, untrustworthy, or spammy 
аз specified in these guidelines, expertise doesnt matter. It should be rated 
Lowest. 





Aren't there some types of 
pages or topics, such as 
celebrity gossip, that always 
have Low quality content? 


For almost any type of page or informational topic, there is a range of content 
quality. Remember that high quality content is defined as content that takes time, 
effort, expertise, and talentskil 


For example, there are both High and Low quality celebrity gossip pages. Often, the 
purpose of these pages is to share scandalous, but potentially true personal 
information about celebrities. We can consider the МС of a gossip page to be high 
quality if ît is accurate and interesting information from a reliable source. 


Pages that are harmful to people or society, untrustworthy, or spammy as specified 
їп hese guidelines should be rated Lowest quality, regardless of their topic. 





I've never seen a High quality 
page of type X. If there are no 
high quality pages of this type, 
why are we giving existing 
pages a Low quality rating? 





For some topics or types of pages, there may not be many (or any!) High quality 
Pages now, but there may be in the future. We need a uniform set of standards that 
apply to all pages, even for pages that have not yet been created. 
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Question = 








‘Some of these criteria seem Ат pages do have a purpose: artistic expression. Pages created for artistic 
unfair. For example, some art | expression do not deserve the Low quality rating simply because they have no other 
pages do not have a purpose. | purpose. Artistic expression, humor, entertainment, sharing photos and videos, etc 
Are these pages Low quality? |аге all valid and valued page purposes. 








IfI have to open the task URL | Yes! More intensive analysis on the website (e.g., researching E-A-T) can be done 
‘on my phone, can I check for |оп your computer. Any time that using your computer will help you complete the 
E-A-T on my desktopllaptop? _ task, feel free to use it. 
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Part 2: Understanding Mobile User Needs 


12.0 Understanding Mobile Users, Mobile Queries, and Mobile Results 


What do you do on your mobile smartphone? 


Check the weather 


Send a text message Cal Mom 
Pace a bid 
Watch a video Read the news 
@ Check updates. 
Set an alarm Pay a game 
А42 Check in for a тот 
| Find a gas station Find an address 
Read a book 
Send a tweet Listen to asong 
Get directions 
П to'a coffee shop 





People rely an their phones for many different tasks in diferent environments. Users may want to search the web or may 
want to tell the phone to do something specific. Keep in mind that tasks can be simple or complex, and may take multiple 
steps to complete. For example, a simple task may be to find the director of a movie. A complex task may be to find a 
movie's showtimes nearby, purchase tickets, get directions, and then use the phone's navigation to go to the theater. 


We expect our phones to do a lot. At the same time, phones can be challenging to use, especially compared to a desktop 
computer or laptop: 


‘© Entering data may be cumbersome: typing is dificult on mobile smartphones, and when users speak to their 
phones instead of typing, voice recognition may not always be accurate. 

= Small screen sizes make it difficult to use some phone features, apps, and webpages. 

= Some webpages are difficult to use on a mobile phone. Website navigation сап be difficult as menus and 
navigation links may be small. Webpages may require left-to-right scrolling to read text. Images may not fit on 
the screen. In addition, many mobile devices cannot access webpages with Flash or other similar features. 

* Internet connectivity can be slow and inconsistent for mobile users going in and out of networks. App 
opening, recognition of voice commands, and webpage load times can be very slow on a mobile phone. 


Important: Mobile smartphones should make tasks easy, even for mobile users with a small screen device (Le. 
size of smartphone, not a tablet). Users want results right away, at that moment, and may not be able to spend a lot of 
time to find what they are looking for. 





In order to do mobile rating tasks, you must have experience using a mobile smartphone, which well also refer to as a 
mobile phone in these guidelines. If you are nat familar with voice commands, device actions, or phone features, please 
take some time to experiment on a mobile smartphone. For example, you can try some of these voice commands: 


* iPhone Siri voice commands. 
* Android voice commands, or say "OK Google” and then "Help" on an Android device 
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121 Important Rating Definitions and Ideas 


SERP for iphone] 


Should | buy an Phone? 


Query [phone] 
Locale: Engish (US) 
User Location: New York, NY 








Query: This refers to the word(s) andlor number(s) that a user types or speaks into a mobile phone. In these guidelines, 
queries have square brackets around them. If a user says "navigate home,” we display: [navigate home]. If a user types 
"iPhone" in the search box, we display: [iphone]. 


There аге many diferent types of queries because users ask their phones to do many things, from opening an app to 
calling a fiend to searching the web. 


User: The user is the person trying to find information or accomplish a task by typing or speaking into a mobile phone with 
a small screen (Le., size of a smartphone, not a tablet). Keep in mind that users are people from all over the world: 
people of all ages, genders, races, religions, political affliations, etc. 


User Intent: When a user types or speaks a query, he or she is trying to accomplish something. We refer to this goal as 
the user intent. 


Locale: All queries have a locale, which is the language and location for the task. Locales are represented by a two-letter 
country code. For a current list of country codes, click here. We sometimes refer to the locale as the task location. 


User Location: This tells us where the user is located, and should be inferred from the map provided. 


‘Search Engine Results Page (SERP): The page a search engine shows after a user enters а query in the search box. 
The SERP is made up of result blocks. 


Result: We wil use the word result to refer to the result block and the landing page. 


« Result Block: This is an individual "block" that appears on the user's phone in response to the query. The result 
block may display information in the block itself or contain links, or may do both. 


* The Landing Page (LP) is the page you see after you click a ink in the result block. 


Device Actions: Mobile phones and other devices can respond to voice commands to perform many actions, such аз 
setting an alarm or opening an app. This is а specific type of query that we'l refer to as a Device Action query. 


‘© Dovico Action query: Specific type of query where users ask their phone to perform an action. These are 
frequently spoken commands used to complete actions that would normally require interaction withthe screen ог 
the device's controls. 


‘© Device Action result: The phone may respond to a Device Action query by performing an action, such as 
calling a phone number, ete. 
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122 Understanding the Query 


Understanding the query is the first step in evaluating the task. Remember, a query is what a user types ог 
speaks into a mobile phone. 


‘Some tasks include a query research link, which you should use if you don't understand the query or user 
intent. Otherwise, please do web research using Google or an online dictionary or encyclopedia. If you still 
don't understand the query or user intent, please release the task. 


Important: f you research the query on Google, please do not rely оп the top results on the SERP. A query may have 
other meanings not represented on Google's search results pages. Do not assign a high rating to a webpage just 
because it appears at the top of a list of search results on Google. 


Think about users in your locale typing or speaking the following queries into heir phone. 

















Query Likely User Intent 
[population of paris], English (US) | Find the current population of Paris, France. 

[starbucks near me], English (US) | Find the nearest Starbucks location. 

[weather], English (US) Find weather information in the user location right now. 

[сай тот, English (US) Calli the number stored for the contact “Mom” on the device. 














123 Locale and User Location 


All queries have a task language and task location (referred to in rating tasks as the "Locale". The locale is important for 
understanding the query and user intent. Users in different locations may have diferent expectations for the same query. 
Each rating task wil show you the User Location information: some tasks have an approximate user location area and 
some tasks have a very specific user location. 


Note: Examples in the following sections wil include a User Location in the form of a city and state. Note that in the actual 
rating tasks, you will need to infer the User Location based on the map that is provided, as discussed in Section 29.0. 


= $ 


Quay football Query football 
Loca Engish (UK) Locale Engish (US) 


in © 


For many or most queries, the user location does not change our understanding of the query and user intent. Here are 
some examples: [facebook.com], [pictures of kittens), [distance between the earth and the тооп], [call mom]. If the task 
does not display a user location, please evaluate it as a query where the location does not matter and use your judgment. 


When is the user location important in understanding query interpretation and user intent? Please use both web research 


and your personal judgment to answer this question. Ask yourself, "Would users in one city or country be looking for 
‘something different than users in another city or country?" 
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124 Queries with an Explicit Location 


Sometimes users tell search engines exactly what kinds of results they are looking for by adding the desired location in 
the query, regardless of their user location. We'll call this location inside the query the "explicit location." The explicit 
location makes queries much easier to understand and interpret. 


My mom is coming to 
visit. 1 need to find a 
hotel for her nearby. 


Query: [Dallas hotels] 
Locale: English (US) 
User Location: Dallas, TX 
Explicit Location: Dallas 


1 need a hotel for my trip to 
New York. 


Query: [New York hotels] 
Locale: English (US) 

User Location: Dallas, TX 
Explicit Location: New York 





‘Sometimes the explicit location matches the user location or locale, and sometimes it doesn't. 


When there is an explicit location in the query, pay attention to it! Users use explicit locations to indicate exactly what they 
are looking for. 


125 Queries with Multiple Meanings 


Many queries have more than one meaning. For example, the query [apple] might refer to the computer brand or the fruit. 
We will сай these possible meanings query interpretations. 


Dominant Interpretation: The dominant interpretation of a query is what most users mean when they type the query. Not 
all queries have a dominant interpretation. The dominant interpretation should be clear to you, especially after doing a 
litte web research, 


Common Interpretation: A common interpretation of a query is what many or some users mean when they type a query. 
А query can have multiple common interpretations. 


Minor Interpretations: Sometimes you wil find less common interpretations. These are interpretations that few users 
have in mind. We wil сай these minor interpretations: 
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Query: [apple] 
Locale: English (US) 


Computer company Frat Person's name ciy 


$e 3 & 


di =й. 
d =й, 
zd 
=й, 


ФТ??? 


Dominant interpretation Common interpretation Minor interpretations 
Query: [mercury] 


Locale: English (US) 


Planet Chemical element Car brand ^ Insurance company Boat engine company 


g АА. mencu 


tt th t og 
тт тт 


Common Interpretations Minor Interpretations 





126 Query Meanings Can Change Over Time 
Remember to think about the query and its current meaning as you are rating. We will assume users are looking for 


current information about a topic, the most recent product model, the most recent occurrence of a recurring event, et. 
unless otherwise specified by the query. 
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41 president 43 president 


LA 
User in 1994 User in 2004 
Query: [George Bush] Query: (George Bush] 
Locale; Engish (US) Locale: Engish (US) 
iPhone 1 iPhone 6 
= Gull 
User in 2007 User in September 2014 
Query: phone} Query. iphone] 
Locale: Engish (US) Locale; Engish (US) 


The interpretation of the query [iphone], English (US) has changed over time as new iPhone models are released, The 
first Phone was introduced in 2007. Users searching for iphone], English (US) at that time were looking for the new (at 
the time) first Phone model. Most users now are looking for the most recent or upcoming iPhone model. Inthe future, 
new models will come out and the dominant interpretation will change again. 


127 Understanding User Intent 
It can ba helpful to think of queries as having one or more of the following intents. 


Know query, some of which are Know Simple queries 
Do query, some of which are Device Action queries 

Wobsito query, when the user is looking for a specific website or webpage 

Visitn-person query, some of which are looking for a specific business or organization, some of which are 
looking for a category of businesses 





12.7.1 Know and Know Simple Queri 





The intent of a Know query is to find information on а topic. Users want to Know more about something 


Know Simple queries are a special type of Know query. Know Simple queries seek a very specific answer, like a fact, 
diagram, etc. This answer has to be correct and complete, and can be displayed in a relatively small amount of space: 
the size of a mobile phone screen. As a rule of thumb, if most people would agree on a correct answer, and it would fiin 
1-2 sentences or a short list of items, the query can be called a Know Simple query. 
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Know Simple queries may be questions such as [how tall is barack obama]. Frequently, Know Simple queries do not 
have question words. For example, [barack obama height] has the same user intent as [how tall is barack obama], but is 
notin a question format, 


Most queries are not Know Simple queries, such as: 


Broad, complex, and/or in-depth informational queries that do not have a short answer 
‘Ambiguous or unclear informational queries 

informational queries on controversial topics. 

Informational queries with no definitive right answer" 

Queries where different users may want different types of information, ог different sources of information 


Here are some examples where the Know Simple query asks for a simple fact, which can be answered correctly and 
completely in a small amount of space, and the Know query answer is more complex. 





Know Simple Query Know Query Explanation 





The Know query is a broad information query and different 
barack obama] users may be looking for different things (e.g., biography, 
books, social madia posts, ete.) 


[barack obama height] 
how tall is obama] 








The Know query is a broad information query and different 
new york city population 2013] | [new york ety ‘users may be looking for йегет! things (e.g., tourist and tip 
planning information, facts, photographs, history). 





The Know query is a broad query for medical information 


hpg oe disease named | graves disease] and diferent users may have diferent needs. There is no 





ael single "answer for this query. 
The Know query is a broad query for a particuiar service 
[macy's store hours] macy's git wrap options} offered by a department store, and does not have a short 





[Wha is the symbol for he 
element nickel 


The Know query is a broad query and there is по short, 


(hat nicke is used fer] аад 








[who won the 2014 bes national | [who ts going lo win he bes | The Know query asks for an opinion and there is no. 
championship game] national championship game] | definitive answer. 
Even though the Know query is theoretically а yosino 
{what is starbucks stock price] {should iinvest in starbucks stock] | question, there is not a single answer that everyone would 
agree оп. 

















Raters must think about mobile users when deciding if queries are Know Simple. Use your judgment here. 


Important Rating Example: The query [weather] may seem like a broad information query, but most mobile users likely 
have a fairly simple informational need: to find the current or upcoming temperature, and the chance of local weather 
events, such as rain or snow. Therefore, we will consider queries like [weather], [weather today], [weather tomorrow], 
[weather this week] to be Know Simple queries for mobile phone users. 





12.7.2 Do and Device Action Queri 





The intent of a Do query is to accomplish a goal or engage in an activity on a phone. The goal or activity may be to 
download, to buy, to obtain, to be entertained by, or to interact with a website or app. Users want to Do something. 
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Неге аге some examples. 

















Query Likely User Intent. 

[get candy crush game] Install tha Candy Crush game. 
nina personaly tes Take an onine personality test. 
wha is my ыт] Calculate BMI (body mass index). 
Buy citizen kane dvi] Purchase this DVD. 














Device Action queries are a special kind of Do query. Users are asking their phone to do something for them. Users 
giving Device Action queries may be using phones in the hands-free mode, for example, while in a car. Its very 
important for mobile phones to accommodate Device Action queries, and we have a high standard for rating these 
results. 


А Device Action query usually has a clear action word and intent. The verb or action ward is often at the beginning of 
the query, but a query might start with "OK Google" or "Google" or “Sir or “I want to.” Use your judgment. 


Here are some examples of Device Action queries. 

















LIIS EES 

EEE [гй monis mobi phone lace а tali lo ante red ok godes yol phas al my 
ЕА 

pe) end a tant toe ami, ttn aa [ите ph ama 

етар татвар осете зано (show me rfl how me aap] 

[Ope an apy orwebpege or — |е алатот аат [open сараас ари ope nebo cni open a funes acm o. 

Peat ana peere are foe ct eaa 








[play me songs from the white album], [watch the Ме of pi [show me cute kittens}, [î want to 
Took at оле ktons] [i want to listen fo imagine dragons radioactive] 


‘Schedule a meeting (schedule a meeting for Sam] 


Play or view media 








‘Set an slam or timer [set an alam for Gam], falar for Gam, [countdown timar for 30 secs] 





Here are some examples of queries that are not Device Action queries. 























Query Query is NOT a Device Action Query: Explanation 
[phone number empire state | There is no action word. The user may be lacking forthe phone number without the intention of 
building) сайпа the number. Well consider this a Know Simple query. 
facebook сот “There is по action word, such ав “open” We will consider this a Website query. 
amazon.com] “There is по action word, such as “open” We wil consider this a Website query. 
“There is по action word, such as "cal or “text” We will consider this a Know query for 
fanno jones] formation, 





127.3 Website Queries 


The intent of a Website query is to locate a specific website or webpage that users have requested. This single webpage 
is called the target of the query. 
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Опе type of Website query is a URL Query, which can be: 


‘© Exact, perfectly-formed, working URLs, such as [йрим от сот] or [wuwibm.com] or [ibm.com]. 
‘© Imperfect URL queries: Queries that look like URL queries, but are not ‘working URLs". These URLs do not load 
if you type or paste them into your browser address bar. Even so, we believe users have a specific page in mind. 


Here are some examples. 
































Query Likely User Intent. 
Kayak, English (US) View the Kayak website. 

youtube] English (US) View the YouTube website. 

bey, taian (Т) View the Italian eBay website 

ew york time health section], Engish (US) View the Health section of the New York Times website 
Iranon com воз digital camera], English (US) | View the EOS digital cameras page on the Canon website. 
12.74 Visitin-Porson Queries and User Location 





Users сату mobile phones with them throughout the day, for example, at work, to school, to restaurants, or running 
errands. One reason to carry a mobile phone is to have help with Visit-in-Person queries, such as finding coffee shops, 
gas stations, ATMs, restaurants, etc. Because mobile phones are often used for Visit-in-Person queries, make sure to 
Consider visitin-person intent as а possibility for mobile phone users. 





‘Some queries clearly "ask' for nearby information or nearby results (e.g., businesses, organizations, other nearby places). 
‘Some queries are not asking for nearby information or nearby results. Here are some examples. 





‘Queries with Visit-n-Person Intent. ‘Query with Non-Visit-in-Person Intent 





How tal is Tom Cre? 


feet 
scan Erich 087 Ошу tow tat is om rl 
Loca Engien [US] 








та | oul ke to take an 


need fed тн eo 


siston ofr ana 





ver ost) 


Jury [gs tons toca: Ene (US) 


Locas Eng (US) 








More examples: [boston red sox], [vashington post, [scrabble 
cheat [definition of sedentary), [aapi], (beyonce), [angry birds] 
[small dog breeds}, [dance videos], [oscars 2012], [pick up ina], 
[bank of america login] 


More examples: [pizza]. yoga class], [cofee shops, [movie 
‘showtimes, [car repair]. [dentists], bank of america atm 
locations, starbucks near me] 














Copyright 2021 т 


Апа some queries could go either way. Some users may want nearby results and olhers may not. 


Here are some examples of queries with both visti 





[hotels] 
[post office] 
[apple store] 
[clibani] 

[best buy] 

Toffice depot] 
Target) 

Tibra] 

Tank of america] 
Tine gap] 


Sen 





1 want lo goto a 
Walmart near me 


Query [waman 
Locale Ena (US) 





| want to shop onina on 
е Walmart veste 


Query mari 
Locate: English (US) 





‘person and non-visit-in-person intent: 
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‘Sometimes, the user location can change our understanding of the query. For users close to Sunnyvale, California, the 
query [turmeric] could have two different interpretations: а popular restaurant named Turmeric or the spice turmeric. 


pep 


T Whats the address of 
* Turmeric? 





Query: [turmeric] 
Locale: Engish (US) 
User Location: Sunnyvale. CA. 








1 want to learn about 


a the spice turmeric. 





S 


= Query: [turmeric] 
3 Locale: English (US) 
User Location: Sunnyvale, CA 





In most other user locations, there is no restaurant (or anything else) named Turmeric and there is [ust опе interpretation 
of the query [turmeric]: the spice. The Sunnyvale restaurant is not wellknown outside of Sunnyvale, California. 





| 1 want to learn about 
the spice turmeric. 


Query: [turmeric] 
= Locale: English (US) 
= User Location: Lincoln, NE 





Use your common sense when thinking about queries and whether they have possible visit- 





person intent 
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12.7.5 Queries with Multiple User Intents 


Many queries have more than one likely user intent. Please use your judgment when trying to decide if one intent is more 
likely than another intent. 











‘Query Likely User Intent 
n Depending on the user need and location, users may want to visit the official homepage (Website), 
el directions (Visitin-Person), or learn mare about the school (Know). 





inert Most users want to go to a nearby Walmart (Vistin-Person) or view the homepage to shop online 
(Website). Soma or few users may want to learn more information about the company (Know). 











128 Understanding Result Blocks 


The following sections contain examples of different types of queries and results. In these guidelines, please 
У assume that the result blocks and pages are easy to use on the phone, unless otherwise noted. 


12.8.1 Wob Search Result Block Examples 


Web Search Result Blocks typically have a title link, a URL and a "snippet" of text describing the page. For many queries, 
Web Search Result Blocks are the most helpful type of result. 








‘Query, User Location, User Intent мер Search Result Block 





‘Query: [cuisinart food processor reviews] 














User Location: Oklahoma City, Oklahoma 





User Intent: This is a Know query. The user 


wants to find recent reviews of Cuisinart food | || Processor, 12-Cup | Willams 
processors. ‘Sonoma 

bw liamsonoma conv. /euiainarte.. || a wens 
Result: This is a Web Search Result Block * 
that has a link to a landing page with Cuisinart | ||. sige viens e kiehen tore 
reviews. epresents he nex generation of Cuisinart 


food processors _ as Reviens 








‘Query: [broadway tickets] 


User Location: New York City, New York ол; 
Theater Tickets | Broadway com. 
User Intent: This is a Know query or Do sorb 





‘query. The user wants to search prices and/or ocean 
purchase tickets to a Broadway show in New | "= Ses Таче 07 


York City. “Sendra The ining Viel 
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12.8.2 Special Content Result Block Examples 
‘Special Content Result Blocks (SCRBs) appear in the search results page, along with Web Search Result Blocks. They 
are frequently, but not always, the first result on the search results page. 


‘Special Content Result Blocks are designed to show content directly to users on the search results page. From working. 
calculators to playable videos to interactive weather information, these results help users immediately get information ог 
content. Some SCRBs тау also have links to landing pages. 


Important: Please assume that any interactive features work and function properly. Some notes: 


‘© All result blocks are “screenshots” or images of search results with prominent links enabled. Unfortunately, a 
screenshot or image of an interactive result block will not function as it would for a real user. For the purpose of 
rating, please assume that interactive result blocks do function as intended. Try to interact with the result 
block as some links, buttons, or other features may work. 


* There may be a delay between when the rating task is created and when you actually rate the block, causing 
some information in special content result blocks to be a few hours or even days out af date. Stock price ог 
weather informational blocks are designed to give users extremely current and timely information. However, due. 
to a delay in rating time, the information may no longer be accurate. Don't penalize a special content result block 
for being out of date. Assume that the blocks show current information for users, unless instructed 
otherwise. 





‘Query, User Location, User Intent. ‘Special Content Result Block 








weather | + auey 














‘Query: [weather] 





User Location: Chicago, Iinois 
userintnt: Tie sa Know simp qey | | 63 

since mobile users have a fairly simple 

informational need: find tne current ` ~ Speci content 


temperature and chance of rain or snow. The 
user wants to know the weather for the User 
Location 


Note: Assume the block shows current 
information for users. 
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‘Query, User Location, User Intent. 


‘Special Content Result Block 





‘Query: [emma stone movies] 


User Location: 





les Moines, lowa 


| User Intent: This is a Know query. The user 

| wants to get information on movies with Emma 
| stone. 

| Result: n this result block, users can 

| immediately see some popular movies starring 
| Emma Stone, with an option to click on the 

| inks to learn more about each movie. Users 
‘can also swipe to see a list of more movies. 























‘Query: [calories in a banana] 





User Intent: This is a Know Simple query. 
The user wants to find aut haw many calories. 
are in a banana. 


ue 





Note: you can elk he 
a Block and see more 








‘Query: [baseball scores] 


User Locatio 





леа, Kansas 


User Intent: This is a Know query. The user 
wants to find the most recent Major League 
Baseball scores. 





ae Query 


حا 








jor League Baseball 


DETR 





a Special Content 
Result Воск 
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‘Query, User Location, User Intent 


‘Special Content Result Block 





‘Query: [coldplay fix you video} 


User Location: 





low Orleans, Louisiana. 





User Intent: This is a Do query. The user 
wants to watch the music video for the song 
"Fix You" by Coldplay. 


Result: By clicking on this result block, users 
сап play the music video for the song on their 
phone, leam more about the artist/album, etc. 











‘Query: [what is the tallest tree] 





User Intent: This is a Know Simple query. 
“The user wants to know what type of tree is 
the tallest. 





measured at 11561 metres (3 
f which rank it as the wer 
tallest known long ее. 





open e) ita. e ie 








12.8.3 Device Action Result Block Example: 


А mobile phone should respond to a Device Acti 
{query to open an app, please assume that the us 
арр, please assume that the user does not have 


jon query and do what the user is asking. If the block is a Device Action 
ser has the app installed on their phone. If the query is to download the 
the app installed on their phone. 





Query, User Location, User Intent, Result 


Device Action Result Block 





‘Query: [open angry birds] 
User Location: Little Rock, Arkansas 

User Intent: This is a Device Action query. 
The user wants the device to open the Angry 
Birds app so he/she can play the game. 


Result: In this result block, users can click the 
link to open the app. 








Birds 





sepe 
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Query, User Location, User Intent, Result 


Device Action Result Block 





Query: [go to amazon.com] 
User Location: Glendale California 

User Intent: This is a Device Action query. 
The user wants to go to amazon.com to 
interact with the website. 


Result: In this result block, users can click 
the links to go directly to the website. 











Query: [set alarm for 5 o'clock am] 
User Location: Tacoma, Washington. 


User Intent: This is a Device Action query. 
The user wants to set the device's alarm to go 
off at 5:00 AM. 


Result: In this result block, the query has 
initiated the alarm on the phone to be set for 
5:00 AM. 














Query: [send text to mom] 
User Location: Tacoma, Washington. 


User Intent: This is a Device Action query. 
The user wants to send a text message to a 
contact nicknamed "Mom." 


Result: In this Device Action Result Block, 

the query has initiated a text to the contact 

nicknamed “Mom,” with an option to text 
er the work or mobile number. 
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‘Query, User Location, User Intent, Result — | Device Action Result Block 


uy: ыным - © 


User Location: Mountain View, Califori 





User Intent: This is a Device Action query. 
The user wants to call the nearest Best Buy E 
store. 





Result: In this Device Action Result Block, the 
query has initiated a call to the nearest Best A 
Buy location. 

va 














12.8.4 How Device Action Results are Displayed in Rating Tash 





Important note: Users issuing queries actually experience the phone's response to the query, while raters are given a 
description of it. If an iPhone user says "Siri, call Mom,” the iPhone displays a visual prompt to show that it's dialing a 
phone number. In these rating tasks, you will see a description of this action. 


Please also assume that the phone successfully performs the action in a fully satisfying way. 


Below are some addtional examples of Device Action queries, along with the corresponding “action text" displayed in the 
rating task. 





This is what he user sees on his or her phone forthe |g what you (he rater) might see in the rating task 








Device Action query 
set alarm for 30 D 


S 


2:18 PM cue tne zou [5x8] 


Acton Setar] 
Ee ae ime 3887] 








The left side shows what the user sees on the phone for the query [set alarm for 30 mins], while the ight side shows tho 
"action text” that you (the rater) might see in the rating task. The text on the right indicates that the user issued the 
query at 1:48 PM (13:48:00), and the device has set its alarm to go off 30 minutes later at 2:18 PM (14:18:00) 
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“This is what the user sees on his or her phone for the = EERE 
en Ed This is what you (the rater) might see in the rating task 


















famem Thé неч nas We flonase 














The left side shows what the user sees on the phone for the query [open facebook app]. In this case, the phone shows 
an "Opening app" message and then opens the app. The right side shows the “action tex that you (the rater) might see 
in the rating task. The text indicates that the device responded by opening the Facebook app on the user's phone. 




















The left side shows what the user sees on the phone for the query [сай дап], while the right side shows the "action text 
that you (the rater) might see in the rating task. The user wants to dial the number stored for the contact “Dan” on the 
device. This text indicates that the device responded by displaying the contacts phone number and dialing the number. 
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“This is what the user sees on his or her phone for the. = = =e 
ا‎ This is what you (the rater) might see in the rating task 











à. FU 


Target 









I T MM 





Target 


Target 


For some Device Action queries, the device displays a list of options to choose from before it can respond to the 
Device Action query. Here is an example for the query [call target]. The user is shown a list of Target stores in the 
area to choose from. This example shows that the Device Action result taken by the device will Бе to call the desired 
Target store after the user makes his/her selection. 








Name: Adele 
Action: Play media] 























[Jum] 








Here is an example for the query [play adele]. The user is shown a "Play media" type of result. In this case, since the 
‘user wants to play music, the Device Action result shawn is a "Play Music" button to click. Clicking this button wil play 
the specified song if the user has the song on his/her device. If not, clicking the result box wil give the user options to 

hear the specified song online. 
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129 Rating on Your Phone Issues 


We understand that raters using different phones, operating systems, and browsers may have different experiences. In 
general, do what you would do naturally, and rate based on your experience. Here are some additional points to keep in 
mind when you are rating on your phone: 


‘© Ifyou see one of these messages when you open a landing page on your phone: 


о Asks whether you want to open the page in the browser or the website's app: select and evaluate tho. 
webpage. However, if the result automatically opens an app by default based on your phone's settings 
(e.g. some users have set ай YouTube pages to automatically open the result in the YouTube app), you 
should rate your natural app experience—you do not have to change your phone's default settings. 

о Asks whether you want to visit the mobile page or desktop page: it is fine to select the mobile page as 
long as the landing page is the same. Sometimes, the mobile option will bring you to the mobile 
homepage instead of the specific URL in the task. You may need to check that the mobile page is in fact 
the same URL as the desktop page. 


* Please open and look at PDF fles. Your experience may be easier or harder than other users depending on your 
phone and browser, but you should stil open the PDF file and look at i 


‘© Occasionally, you may be assigned some of these rating tasks on a desktop computer, but please rate from tho. 
perspective of a mobile user unless otherwise instructed. 


‘© You should assume queries were issued on a smartphone unless otherwise stated in the task or project-specific 
instructions. 
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Part 3: Needs Met Rating Guideline 

13.0 Rating Using the Needs Met Scale 

There aro many diferent kinds of queries and results, but the process of rating is ho samo: Needs Met rating tasks ask 
you to focus on mobile user needs and think about how helpful and satisfying tho result is for the mobile users. 


і 


МА кем ем ни ғ 
Pom: Мо, Foreign Language No. [Di Nol Lon No 


This is what the Needs Met rating slider looks like: 





Rating Description 





A special rating category, which only applies to certain queries and results. АП or almost all 
Fully Meets (FullyM) |mobile users would be immediately and fully satisfied by the result and would not need to 
view other results to satisfy their need. 





Highly Meets (HM) | Very helpful for many ar most mobile users. Some users may wish to see additional results. 





Helpful for many users OR very helpful for some mobile users. Some ог many users may 
Wish to see additional results. 





Moderately Meets (MM) 





Helpful for fewer mobile users. There is a connection between the query and the result, but 
Slightly Meets (SM) [not a strong or satisfying connection. Many or most users would wish to see additional 
results 





Completely fails to meet the needs of the mobile users. All or almost all users would wish to 


Fails to Meet (Falls) oe 1915 0 me 











Please note that you may assign in-between ratings. Use in-between ratings if you think the rating of a result falls 
between two labels. You can either drag the slider or click on the point that you want the slider to land on. 


13.1 Rating Result Blocks: Block Content and Landing Pages 


For Needs Met rating, you will assign a rating to each result. Each result includes the content inside the result block and 
landing pages associated with the result. 


Which part of the result do you rate? It depends on both the query and the result block: 





уре of Block. What to Rate. 





“The content inside this type of block should always play a large role in your rating 
‘Special Content Result Block 














(SCRB) Some Special Content Result Blocks may have links to landing pages. In these 
‘cases, think about whether a user would click on the link in order to satisfy their 

Note: Assume that interactive | user need. 

‘result blocks function as intended. 

Try to interact with the result block | = Іг most users would not click, rate the Special Content Result Block based 

аз some links, buttons, or other оп the block content alone. 

features in your rating task may æ If some or many users would click, you may consider the helpfulness of the 

work landing page(s) in addition to the content in the block. In this case, both 

need to be helpful to justify a high rating. 
Wob Search Result Block А dick is required, and you should evaluate the landing page to assign a rating 
Device Action Result Block ^ Base your rating on the helpfulness of the action itself. 
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For example, think about the query [what does love mean}, 





Result Block 


‘Query: [what does love mean] 


Rating 





‘Special Content Result Block 


love 











Most users would probably not click on 
the Special Content Result Block, 
because the black contains a large 
amount of helpful content and has no. 
obvious landing page link associated 
with it 


Therefore, base your rating on the 
content inside the block itself. 





Web Search Result Block 








What ls Love? We Spoke to the Mariam- 
"Webstar Laxieographer 

suenan: швеи sch a отра 
ieza seems йе аана, 








In this case, users would have to click on 
the web search result in order to gat an 
answer to the question. 


Therefore, base your rating on the 
content of the landing page. 
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Here аге some examples of Special Content Result Blocks where the block should be rated primarly on the content inside 


the block itself. As always, please use 


your judgment. 





‘Query and Special Content Result Block 





Query: [utopia animal hospital] 





ا ا 
REOR 5‏ 
Utopia Animal Hospital‏ 

т 9 





Query: [chicago weather] 


Chicago, IL 


210" 





Query: [calories in a banana] 


105 calories 





Query: [now to find security code on 
visa] 
The cord security code 
is located on the back of 
MasterCard, Мез and 
Discover credit or debit 
cards and is typically separate 
‘group of 3 дуз 1o the right of he 
Signature stp On American 
Express cards, пе card security 
‘code в а printed, not embossed, 
group of four ев on the front 
‘wards te rignt 
Care еситу ease Wipes, te төө 
КЧ 
луу ма нани cot 





Query: [no did Lincoln die] 


Assassination 




















Query: [movies san francisco] 


Movies near San Francisco. 











Note: The guidance in this section specifically applies to Needs Met ratings. For Special Content Result Blocks that have 
landing pages, you may or may not also be asked to provide Page Quality ratings. In these cases, your Page Quality 
rating should always be based on the landing page. Please refer to Section 14.0 for more about the relationship between 


Needs Met and Page Quality ratings. 
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132 Fully Meets (FullyM) 


Fully Meets is a special rating category, which can be used in the following situations: 


* The query and user need must be specific, clear, and unambiguous. 

* The result must be fully satisfying for mobile users, requiring minimal effort for users to immediately get or use 
What they are looking for. 

= Al or almost all users would be completely satisfied by the result—users issuing that query would not need 
additional results to fully satisfy the user intent. 


In other words, the Fully Meets rating should be reserved for results that are the "complete and perfect response ог 
answer" so that no other results are necessary for all or almost all users to be fully satisfied. 


You will need to use your judgment to decide whether a result block can Fully Meet the user need. Here are some 
scenarios when the Fully Meets rating is appropriate: 


* The user is clearly looking for a specific webpage or website and the result block with the specific webpage ог 
website Fully Meets the user's need. 

‘© The user is trying to complete a device action and the result block accomplishes the desired request. 

‘©The user is looking for a very specific fact or piece of information and the result block provides the information 
immediately, thoroughly, accurately, and clearly. No other results would be needed. Before using the Fully 
Meets rating for queries seeking a very specific fact or piece of information, you must check for accuracy and 
confirm that the information is supported by expert consensus where such consensus exists. 


Fully Meets may apply in other situations as well. Ask yourself whether the result block alone is the perfect and complete 
result that would fully satisfy ай or almost all users. Be conservative when using the Fully Meets rating. When in doubt, 
consider a lower rating 


Note: If a result lock is very close to being fully satisfying, but the block alone may not be sufficient in order to fully satisfy 
all or almost all users, a rating of Highly Meots* may be appropriate. 


Sometimes you have to think about how specific the user need is and will need to make a judgment call on whether it is 
specific enough to possibly have a Fully Meets rating. 


13.2.1 Examples of Fully Meets (FullyM) Result Blocks 














(Query and Userintont — Result [Rating Fully Meets Explanation 
— 
— م‎ 
m— = ar 
jetadis ЕСЕТ 
И ГЕЯ reser et vere is opening the specified website. 





User Intent: Go to 


ыша URL: p wwcamazon сопу 


Website Name: Amazon.com: 
Online Shopping юг 
Electronics, Apparel 
Computers 
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User Intent: Go to the Yahoo 
website. 








‘Query and User intent | Result Rating Fully Meets Explanation. 
Query: amazon] Te query has clear intent 1o go 
9 тоба amazon com webste 
User Location =e yg ini tas may be oiher 
d лена, „бае тешеп ms | fM м interpretations for the query, the 
eir umen pissed ied 
User intent: Go to the е; 
Amazon website. pe 
тәти Exec tor Pay Less 
Query: pargetwebste) Танке 
PII 
User Location: Jacksonville, | шшр шр Ел Te query has clear intent 1o go 
ج‎ io ayet zm. 
n— 
peter 
‘Query: ehop почват 
оше] 
Use Location: Morro E — À.. menie has caritate go 
т 4 ia ihe Norton өере. 
User intent: Go to the 
Nordstrom website for online. 
shopping, 
‘Query tanc mab] 
Titanic - IMDb 
etan | am | rr Te query has clear intent to go 
ne Moskmendy-Tume-imeremelmes | UO ® ш to the IMDb page for Titanic. 
User Intent: Go to the IMDb | panes annie cat 
page for Тале 
Query: fw yahoo.com] 
Yahoo Even tough tss an imperfect 
User Location: Denver, | 18100, 
ечат LU Me aaeeme s | ggg [URL quan: Ке clear the user 


anis to go to ће Yahoo 
website. 





Query: [enn heath) 


User Location: Annapolis, 
Mayland 


Health News - Medical 
Mental and Dental 





The query has clear intent to go 
"the Health section of he 





User Intent: Go tothe CNN 
website. 

















ТЯ a.m webpage. 
User intent: Goto the Heath | LDR cur 
Section of enn com. 
‘Query cn] 
is The query has clear intent lo 
User Location: Arnapots, | YET access CN news and contant. 
Maryland оло ашнен P: ‘The website has һе content the 


user is searching for, and fully 
salis the user intent. 
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‘Query and User Intent — [Result Rating Fully Meets Explanation 





This result Fully Meets the user 
aed whether they want the Yelp 
app or the Yelp website, which 
are both popular. 


Nota: If you see this kind of 
result, assume that the user has 
Query: рер ‘the app installed on their phone. 


User Location: Raleigh, Clicking on the Yelp tink above 





North Carolina асас | he logo opens the арр directly, 

nE | and etcking on the Open on 
User Intent: Open the Yelp po yelp.com’ link gives the option 
app or go lo the website at af opening the app or website. 
yelp.com. 












Query: lebron james stats 














[oriens M 
mum Ime suyas tact ge 
user Loca TT Dine doje tats pages 
MnP Eramaten me cam, | al [lemon ames ena spac 
AENT em. 
eer iten asta peer | Ои абы uaria ok 
аатта сотан EES 
Eme 
What he er aes 





‘Query: [open instagram] This Device Action query is 


lear and specif: the user 





User Location Wat you might ee inthe 
кы Еч eat ve arts 1o opon па tage 
a тшк ir. Theiesutüodk thoes fe 

Tone е оома ot opting 
User intant: Open the Acton Open an в 
Instagram app onthe user's ый "he specified appt Fully 
" Werte pe wet net 
pra көт Nm 








‘Query: [open bing search 


appl The result block shows that he 


phone has detected that the app 
gg |8 по назе and gives he 

‘option of downicading the ap. 
This result Fully Meets the user 
need, 


User Location: Dalias, 
Teras 





User Intent: Open tha Bing 
Search app on the user's 
device, 
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Query and User Intent 


Fully Moets Explanation. 





‘Query: [open facebook apo] 


User Location: Richmond, 
Virginia 


User intent: Open the 





What you might see in the 
rating task: 


This Device Action query is 
ваг and specific: the user 
wants to open tha Facebook. 
арр. The result block shows the 
Phone's confirmation that tis 
opening the app—it Fully Meets 





Facebook app on tne user's e ‘Acton: Open an арр Бары 
App: Facebook 
Query: [set alarm for MEA o This Device Action query is 


o'clock am] 


User Location: Tacoma, 
Washington 


User Intent: Set the device's 
alarm to go of at 5:00 AM. 





What you might see in the 
rating task: 


‘Action: Set alarm 


Arm Time: 500 AM 





dear and specif: the user 
wants to sel the phone's alarm 
‘ack fr 500 AM. The result 
block shows the phone in the 
process of seting the alarm at 
‘he spacifed time—i Fully 
Meets the user need. 





Query: [chef chu phone 
imber] 


User Location: Los Atos, 
Caloris 





User Intent: Find the phone 
number for the restaurant 
сайга Chef Chus, 





V. (650) 948-2696 





Chet Chus is Chinese 
restaurant located in the user 
locaton. This result block 
immediately shows the correct 
phone number with the option to 
al the number directly on the 
device it Fully Meets the user 
өп. 


Nota: You must check for 
accuracy before using the Fully 
Meets rating. You can verity 
"hat he phone number in this 
‘SCRE is correct by checking 
Chef Chis oficial website. 








Query: eal best buy] 


User Location: Mountain 
View, California 


User Intent: Call the nearest 
Best Buy store. Note that 
thera is опу one Best Buy 
store located in Mountain 
View. 


What the user sees: 














What you might see in the 
rating task: 


Acton: Make a call 
Business Name: Best Buy 


Location: 2460 E Charleston Rd, 
Mountain View, CA 94043 


Phone number: (850) 903-0501 








This Device Action query is 
clear and specific: he user 
‘wants to сай the nearest Best 
Buy store. The result block 
shows the phone in tha process 
af calling the store with he 
correct phone number—it Fully 
Meets the user need, 


Nota: You must check for 
‘accuracy before using the Fully 
Meets rating. You can verity 
that the phone number in this 
Device Action block is correct by 
checking Best Buy s official 
website for the location in 
Mountain View. 
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Query and User Intent — [Result Rating Fully Meets Explanation 








Query: [орел map of italy] This Device Action query is 
lear and specifc: the user 
wants to see a map of tly. The 
result block shows a map with a 
link to open tin Google 
Maps—it Fully Meets the user 
eed, 


User Location: Baltimore, 


Maryland What you might see in the 


rating task: 


User intent: Open а map of 


Italy on the user's device, Acton: Open map 


Location: Italy 








This Device Action query is 


Query: [navigate to yosemite clear and specific: the user 


ational park] 





What you might se in the | wants to reveste ta Yosemite. 
User Location: Sat E owe phone o proosas 
Washington Acton: Navigation t opening up he Navigon 





app. which wil give the user 


User Intent: Navigate to Destination: Yosemite National |turn-by-tum directions trom the 


Yosemite National Park from 











eremita National = NL ‘ters location Fully Meets 
a usar nau 
Ж Destination Adres: Yosemite 
Vilage, CA 95369 

БЕ 

The cad sey ce = 

е betel 
сому: how lo бп security | Maca vase Te result block displays tho 
ade on visa) aran ans a py separate complete and correct answer 

oup eia agis a Pe Ton e Prominent in tne res lock, 
User Location: Chicago, | feres O та та way а о easy бг users 
Ninos ires cari, ma ca оу TETUR [on meble devices to rond. The 

Sedo apna тетш oct also includes an maga 
user intent: Find out howto | group siten dots one ent honing usen асаду where o 
locate ihe secunty code on a | trade me nart: find tho сода and he Back 
Visa cara poi Fully Mess bs usor need. 


E. ee 

















Copyright 2021 96 








‘Query and User intent — [Result Rating Fully Meets Explanation 





pat a The result block displays the 
Query: starbucks stock лүн complete and correct answer 
price PSL * prominently in the result block, 

а way hat is easy for users 


User Location: Alexandria, egg [on Mobile devices to read—it 


Virginia №, исиги: (Puy Meets ha user nead. 
User Intent: Find the current Nota: Assume that the result 
sock price for Starbucks. shows current information for 








8.337 million (2012) 


Query: [new york ty arido 


population 2012] The result block displays the 


paper complete and correct answer 
prominently in the result block, 


User Location: Pasadena, = f Fre wy ati anay for sara 





Сайта кк ж T^^ Gnmabile devices to read it 
User Intent: Find the s d Ко eMe in nat nut 
population of New York City | [attra | Ste anes 

Back in 2012. mails 





“The result block displays the 
‹ complete and correct answer 
prominently in the result lock, 
یا‎ hla waaar: їп a way that is easy for users 


forecast for today] 50" оп mobile devices to raadit 
Й 





User Location: Fully Meets the user need. 


Mountain View, Calera. IE Remember tat we consider 


weather querias, Ike this one, to 


User Intent: Find weather bo a Know Simple query for 


forecast information for the TET 
current day in Mountain View, | S A тобе phone users. 
California. 


Nota: Assume the result shows 
Д 3 current information for users. 








The result block displays he 
juery: london time] complete and correct answer 
eRe prominent in tha resul lock, 
User Location: New York, | 11:05 PM in a way that is easy for users 
New York gegen [en mobe devices t re 
a eM Fly Meets te user need. 





User intent: Find the current 


ime in London. Nota: Assume thatthe result 


shows current information for 
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‘Query and User Intent — [Result Rating Fully Meets Explanation 





‘Query: [who is the chancellor 











[of geram PENR The result block displays the 
й voca verkel Е 
оне -— romani in ba re toc, 
2 ET 
User intent: ind tha name 
‘of the current Chancellor of амы, 
шу 
[aue trovon at shoreline 
nore 
To inont isto vist me sects 
ser Location: Mort pas ton speedy the 
Чен attra кетгут телне нне 
кши жшн | tected Cheon ston, win 
eerie Роа ња наа || S „© & ©, оттоо к vl is oci 
ес ato ресей y е репо. 
Serneawicvanne | 





location in person. 








The Crane Wife 3 


luris 





The result has exacty what the 


Query: lecemberits crane user wants, displaying the lyrics 


wita 3 ics] 





to the specified song clearly and 
completely inside the result 

User Location: Long Beach, 

California. —— a block. The user does nol need 





TTT | io cick to a new page to see the 
complete уйе for his song. 





User Intent: Sae the lyrics la | 2 00, 
The Decemberists song — 


caled "Crane Wife 3 Nota: The ric posted on 


Google Play are licensed. 
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Query and User Intent — [Result Rating Fully Meets Explanation 





The intent isto find gas stations 
near the specific user location, 
represented by the lue dot on 
the map. 


This result block has a very 
satistying ist of nearby, popular. 
and prominent options within 
ne mie of tho specific location 
This kind of block is especially 
helpful for users who want to 
eit the business in perso. 


Query: Igas stations near 
mel 





User Location: Mountain. 


View, California This result would be very 


convenent for mobile users who 
need gas immediatly, 
especialy for those who are 
Giving and cannot easly use 
ko "heir phone. 





User Intent: Find gas 
stations near the user 
location. Nate that tis may 
be an urgent query. 








Note: A range is included 
because some users would be 
uy satisfied with these 
Selections, while some users 
would want more information 
(е0. gas pricas, hours, distance. 
‘rom location, longer ist of 
options) 











The intent i to find coffee shops 
naar the specific user location, 
represented by the blue dot on 
the map. 


“This result block has а vary 
чавыупа ist of nearby, popular. 
and prominent options within а 
few miles ofthe speci 
locaton. This kind of Backs 
specially һар for users who 
User Location: Mountain Want to visit tha business in 
View, Сайота EAE КЧ person. Note that hace blocks 
ate ars interactive -eicing ona 
User intent: Find colt ofa shop wil give ceptor to 
shops near the user location. El the business. get rections, 
Faad reviews. ок. 


Query: [nearby coffee shops] 











Note: A range is included 
because some users wouid be 
uly satisfied with hese 
ms selections, while some users 
woud want more information 
(ев. hours, distance from 
location, longer ist of options). 
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‘Query and User Intent 


Fully Meets Explanation 








‘Query: [1600 pennsylvania 
‘ave washington dc] 


User Location: Sakersvile, 
Сайта 


User Intent: Find а тар. 
directions, information about 
‘what is located at his 
address, etc. 











1800 Pens ana Ave 
hw Wannen, De 


This block contains а map with 
tha queried address, link to get 
directions, and specifies the 
landmark at the well-known 
‘address on the map (The White 
House 


к not unusual to search for an 
address (or business) that is far 
from the user location. Here, 
фе user has explcity asked for 
а specific address in a far away 
place, апа this result в very. 
зайыупа. For a query this 
Specife, the user locaton does 
по. change ho rating. 


Note: A range is included 
because some users would be 


fully satisfied with this resul, 
hila some users wouid want 


mara information (e... photos, a 
brief description 





Query: [metro center staton} 


User Location: Adams 
Morgan, Washington D.C. 


User Intent: The user wants 
to see a map, schedula, 
transfor options, andlor other 
rain information for the Metro 
Cantar station, given that the 
user is close to the queried 
Station. 














This block contains a map, ink 
to get directions, departure 

imas for the multiple train ines 
"nat operate at his station, nk 
toa delaled schedule (а he 
"Departure board) and an 

option to see mora information. 


Since the user is located in he 
same city asthe stain, the 
Information in this block is highly 
satisfying. 


Note: A range is included 
because some users would be 
ul satisfied with ns resul, 
while some users may want 
moro information (o... nearby 
alracins, ticket prices) 
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13.22 Examples of Queries that Cannot Have Fully Meets Results 


There are sore queries that cannot have a Fully Meets result. Here are some examples. 





Type of Query 


Example. 


No Fully Meets Result: Explanation 





Broad queries where no 
single result could fully 
satisfy ай users 


[keting] 


This is a broad informational query. Knitting 
is an activity anyone can do and that anyone 
‘can create a website for. Different users 
may want different types of content: videos, 
instructions, pattems, etc. There is no one 
official source for knitting information and no 
‘one result could satisfy most users. 
Therefore, no Fully Meets result is possible 
for this query. 





Famous names 


е... [barack obama] 


This is a broad informational query and itis 
impossible to know exactly what the user is 
looking for 





Ambiguous queries 
without a clear user intent 
or dominant interpretation 





fada] 


There is no dominant interpretation for this 
query. The folowing entities are all common 
interpretations: Americans with Disabilities 
‘Act, American Dental Association, and 
‘American Diabetes Association. While each 
interpretation has an official homepage, 
‘none is Fully Meets since there is по 
dominant interpretation 








Non-famous people names 


e.g. [sam wen] 





‘Queries for people's names can be tricky 
Many or most people queries do not have а 
dominant interpretation. Even unusual 
sounding name queries may not have a 
‘dominant interpretation. For example, the 
‘queries [sam wen], [tran nguyen], and [david 
mease] can have no Fully Meets result 
because there are multiple people with each 
of these names, and itis not clear that most 
users are looking for any one particular 
individual, 
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13.3 Highly Meets (HM) 


A rating of Highly Meets is assigned to results that meet the needs of many or most users. Highly Meets results are 
highly satisfying and a good "ft for the query. In addition, they often have some or ай of the following characteristics: high 


quality, authoritative, entertaining, andlor recent (e.g., breaking news on a topic) 


In order to receive a Highly Meets rating, information pages such as encyclopedia articles and news articles must be 
accurate and highly credible. Highly Meets medical and scientifc information pages must represent well-established 
scientficimedical consensus unless the usar is clearly seeking an alternative viewpoint. For all information pages, you 
must check for accuracy and confirm that the information is supported by expert consensus where such consensus exists. 


A query can have many Highly Meets results. 


Have high standards for the Highly Meets rating. This is especially important for queries with many on-topic results. 


13.3.1 Examples of Highly Meets (НМ) Result Blocks 






































Query and User Intent | Result Rating Highly Meets Explanation 
к is esu shows a compita tat 
Saa oor ne ‘of all three locations in the 
hate wes нн ro tn 
User Location: Chal, Pratis especialy het Tar usere 
А a БЫНА 
Mee Thre ae | cmm x. | ETS [notera resi ok la not 
MG pasate usar ionis: Е Fuly Meets because users who 
ha want to go to the website to see 
yana ey beaten | шу, 
sata rary ee ое pensions i mid 
ái have to see additional results. 
Ti he ica wba or 
vey: trader jos Teater Jace wc hae ly 
ойша Celo on ho 
user Location: chat, канен sen pea, 
Кл Ts 
User Intent: There are Е ишы Note: This result block is not 
wo possible user intents: | formen Fully Meets because users who 
trot usere probe war. чан ютап val he мше 
E vara neat caton or Rave doa bor wrt tad 
К-П iat orion ет 
redi aval 
ive siot 
modem art) mi is the museum's oficial 
n e rh 
ne Naw Ni ota | Museum ot Modem art ‘membership, the store, et. 
[көтүе РТТ. (Nota: This rut lock rot 
ое оне Fully Meets because users who 
trot usera probe want 
ман ematen ш ыг he 
va hem ce vantinernaton to vel he 
[fao uaar in located in to find that information —1 is not 
Feit pay rosa immediately available. 
cata or go o e 
oa, 
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‘Query and User Intent | Result Rating Highly Meets Explanation. 





‘Query: [museum at ED 


modem ar 3 The result lock shows 


information about he museum, 
and is especialy helpful for users 
who want te visit the museum. 





User Location: # танна 
‘Manhattan, New York 





ote ut oc ot 
тИ |ы. STU CAES, [Patr Meets oven hough there 
оу he museum ше | уш Фабула fr users who want lo 
Beier bcati Sotto neta tecas i 
Newa te saute | me эс orate a a vl 


Search resul block wih ate 
bones orao a he поа nippet, and URL. 

















“The query is the name of a car 





^ dealership in New Jersey, which 
is the only dealership with that 
Mo ame in the user location 
Query: [prestige bmw] (Ramsey, New Jersey). The 
result block shows information 
User Location: about the dealership, and is 
Ramsey, New Jersey especially helpful for users who 
——— Want to visit he business. 
User Intent: There a М e © (jux xw 
‘wo possible user intents: ^ = ° Note: This result block is not 
most users probably want Fully Moets— even though there 
{visit the car dealership | 9 wus aes uerus is a website link, itis not very 
or goto the website 9 , У Satisfying for users who want lo 





о1о the website because itis 

бо as informative as а web 

К ratus ‘Search result block with a tile, 
snippet, and URL. 








This в the car dealership 
oficial website and has very 
helpful information about. 


‘Query: [prestige bmw] 


User Location: 
Ramsey, New Jersey pamm inventory, fnancing, ec. 
User Intent: There a о миңине лени ee | nis 3 V н | Note: This recut Block is not 








Fully Meets because users who 
want information to visit the 


dealership have to do a bit of 
‘work to ind that information tis 


not immediately avaiable. 


‘wo possible user intents: 
most users probably want 
то visi tne car dealership 
ar go to the website. 
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Query and User Intent 


Highly Meets Explanation. 





Query: [belmont ibrary] 


User Location: Belmont, 














There is опу one library in the 
user location (Belmont, 
Massachusetts). The result 
block shows information about 
‘he ibrary, and is especialy 
helpful for users who want to visit 
the locaton. 








аала 
TETE iota: This result block is not 

Userintent: Fna ا ا ا‎ a 
® а неба Ink, snot very 

arare ge e satisfying for users who want lo 
сни go to the website because it is 

С Pots потап as а web 
z earet veut boc wi ae, 
Snippet and URL 
Р 5 ie There is only one library in the 
AM // user location (Belmont, 

tee / cl California). The result block 

Query: [belmont library] жа керг 
User Location: Belmont for user who want to visit the 

iocis caton. 
ee تي‎ 

жилы Note: This result block is not 

[тууйт Ө Mim - Fully Meets even though there 
een Кы iS a weeks ink e no very 

arr goto he ө ите СЩ 
O a mmm [goto е website because itis 





Pot as informative as a web 
Search result block with a tile, 
Snippet, and URL. 





Query: [decemberists] 


User Location: Long 
Beach, Саната 


User Intent: Leam more. 
about ihe band, The 
Decemberists. 





‘The user intent is too broad to 
have a Fully Meets result 
However, this is The 
Decembersts oficial website 
and has a lot of content including 
our dates, music videos, the 
latest album, etc. The result is 
uniquely authoritative and most 
users would be very satisfied by 
this website. 











Query: [cone dion] 


User Location: Bellevue, 
Washington 


User Intent: Learn mare 


about the singer Celine 
Dion. 














The user intent is too broad to 
have a Fully Meets result. 
However. this is Celine Dion's 
official website and has a lot ot 
content incuding recent news, 
how information, music, photos, 
‘videos, ete. The result is uniquely 
authoritative and most users 
‘would be very satisfied by his 
website. 
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‘Query and User Intent 


Result 


Highly Meets Explanation 








‘Query: ear the walking 
deas] 


User Location: Mountain 
View, California 


User intent: 
Find information about the 
TV show (e.g. show 
summary, episode guide, 
cast information, 
‘steaming options, atc 





‘The user intent is too broad to 
have а Fully Meets result, and 
diferant users may ba looking for 
different things. 


This interactive block has a large 
‘amount of helpful content for a 
TV show including a summary, 
episode and cast information, 
lc iis result would be vary 
helpful for many or most users. 





Query: [manresa reviews} 


User Location: San Jose, 
California 


User Intent: Find reviews 
for this restaurant located 
im Los Gatos, Саната, 


Manresa, Lon Gatos - Menu, Prices & 


eves ТФА 


Tae ea 30 











‘The LP for this web result has 
127 reviens tor he restaurant 
and is easy to use on a mobile 
Phone. The first three reviews 
Show automatically. and then you 
сап cick o seo more. This 
result would be very helpful for 
many or most users. 





‘Query: [who is tho. 
‘chancellor of germany] 


User Location: New York, 
New York 


User Intent: The user 
wants o find the name ol 
the curent Chancellor of 
Germany. 





озонго Germany: Wiüpeda the 


aD 











و 











This в a specific Know Simple 
query foc the name of tha current 
Chaneallor of Germany. Wie. 
the answer is inthe description 
ofthe web result is not 
‘splayed prominently in a way 
that is vary easy for mobile users 
to read. 
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‘Query and User Intent [Result Rating Highly Meets Explanation 
Remy ному ки The LP for this web result offers 
any Broad cial fr saie, 
User Location: New York, | rosdvay tickets show dates {roma highyiusrthy sours. 
New York Бо ИРИ ae ad 
پس‎ Ж» sra to navigate he ste т 
esr intant: Faden result would be very helpful for 
Созду оен many or most users because iis 
Панса Broadway золу Wish 
bes 
The L fortis пое атау 
[yn res ostera many ki 
Backpacks ке аве Tha 
сому: ids backpack] сопу a venae or 
er Location: Now Yor, | ‘Si Ss pce Menge тапсы апа seting high 
[user Location: New York, | Шу с Spana nam 77 quality backpacks for kids of 
ow Yor | SSS, [dient agis There tat ct 
[unir iso dai Е Proc emn and тапу 
et: Ree. tna reviews or each Басра, 
tees ent ee in addition to fiter options that 
ма ess to um ота pone. Tis 
паи нола be very пары tor 
many or turers 
Query: poison му The Loris mobie-fiondiy 
web raul par ofan 
User Location: Ananta, | outnacing puisen ivy ana ter mortals соттан vaheta 
Georgia eho Pate and provides alt f hal 
наана ЖШ ЖД. |кютәюп about poison ivy. 
User intent: Find pictures | Sc ca ‘hs ros woud be very пары 
prr MEL Tr ry reir Men or 
meta рон! tow Some ners may май to see 
fret paisan ivy e icona reni 
сому: ich oan 
er Location: Boson, | Matt dudan -napon te ee тер of this web eut ва 
UsarLcaion nosion, | ete blend Wipe ate 
Po ree a ZEE, | about Michael Jordan. This 
нире ерт fnt им! ar any 
raton. news, image, 
c about Michael Jordan 
Users are looking for good or 
query: lance vidc] riri danca vaca and 
Белото Dance - YouTube Pe ua any Spiane ona 
User Location: New York prinio А 
Мен York > pe zi. | Thera are many on-topic results 
eri qu end tants ona very 
tsar intant: Find a dance орг video ot a comedian 
Video wach Garang dance sar tom 
evi acids 
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Query and User Intent [Result Rating Highly Meets Explanation 
Query [reste trol 
User Location: Colorado 
pings, Colorado sexing magos of ruste bridge 
ку hp и шемер 
er intant: Tha user Троа наре nine css 
мала tounderani what ; ИЕТ 
rasna bridge orm Ы 
Бн may bà more һар hana tt 
mem (2g. wrati oks esr due he uie 
o hv йау ar t, es of he ge 
sect mata, 
жонс rari, 
Ж) 
Query buie 
раз] 
тыа мод conti many het 
genes : =p, [eges ad a ecg pae 
TETUR ET osse mire mage 
(ИДЫ Puy Meets шо ont 
ror китай about 
pute conve 
Anite pcs 
ery лане spei] | Herrera and 
uem Tho regu block has a good 
er Location: St Loui, | “шее онсе PM umma ot what imde 
Misson O isi ne gy [spe re. but soma sere 


User Intent: Find more 
information about invasive 
species, 





Ка 





‘would probably want to see mora. 
Information like examples. 
impact, ete. 





Query: [turmeric] 





User Intent: Find 
information about he 
Spice or he restaurant in 
Sunnyvale 





Tuer Restaurant 











3 











For users in Sunnyvale, 
Calitomia, the query turmeric] 
саша have two diferent 
Interpretations: a popular 
restaurant named Turmeric or. 
the spice turmeric. Because the 
user is focaled in Sunnyvale, 
‘many mobile users are probably 
locking for the restaurant 
However, itis not completely 

clear because some users may 
want to find information about the 
spice. This result about he 
restaurant has salsying 
information for users wno want to 
visit —it Highly Meets the most 
likely user need. 
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‘Query and User Intent 


Rating 


Highly Meets Explanation. 





Query: [ow to get from 
seat to xppzyhi] 


User Location: Seat 
Washington 





User intent: Get 
directions from Seate ta 








Even though the user has to 
re-enter the destination, the 
result block recognizes hat he 
usar was asking for directions 
fom Seale. The user has to 


enter the destination manually 
because the name in the query 
Crpppaytr) в so garbled that 





























hor lace, Note at CECI 
бла коооп ба query rae к n good guess aao 
eset) ie garbled and t a uer tet 
енше to know 
al te user vanta. nice this block в ating 
The rame wis йау Because tiheine a user corar 
шүрли arabe е request ала mon immediatly 
а voca quay) Sot ке died diecione 
сону: iden wiat 
тив в aan sia dedicated io 
user Location: New Yor, | QVI crimen Kision Wig. The welete has 
wade monne отита iato cing 
р, |r 8.000 pictures and 300 
User Intent: Find more „= ш lw | video clips, interviews, articles, 
‘information about the and more. This website Highly 
actress, comedian, writer, | Meets the need of many or most. 
land producer ps 
сону: ores alphabet 
тив is a broad query. Diferent 
User Location: Alban бше may ба lohan for rant 
Now Yor вее 
User intent: Fia gg | Ties shows matic tatios 
historical information about T = "^" | of images with the letters in the. 
o Grook alphabet Ve rook арлабы. та resu 
raves Ге ais in he ош! ba hp ке many users 
Greco habat images c Serna mere may wish о ива 
he enere, ec аала rest 
тив в a broad query toran 
at nce Ste ie инн actress. Difrent users may be 
‘Query: jennifer aniston | ENTER "Я locking for different things. 
ا‎ | oe тп rit shows recent newe 
Dewars н tout Jen Arion. Tha. 
Ses : |, |afides are timely (at he tme 


User Intent: Find images, 
recent news, gossip, 
information, etc. about 
Jennifer Aniston. 




















"bis was writen) and relatively 
Interesting, and not just common 
‘everyday news about ће 

actress. Many users would be 
Satisfied wih the result and some 


users may wish to see more 
results, 
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‘Query and User Intent | Result Rating Highly Meets Explanation. 








This is a broad query for a city. 


p Dicet users may be койор 
Query: seate, е і for diferent tings. 

















washington =; 
E The result contains a wide 
User Location: Апарт, | = variety of help information 
Texas ogee ‘ Sich as a map, weathar 
pains MAURO S. АЙ 
User intent Find E Ф eres. апа has ал option о see 
infomation. news, maps, | ИИД ‘mara infomation containing 
te related о Saatte. 2 Фл Nearby event and facts about 
Washington = {he cj. Moderately Meets” to 
Highly Meets isan appropriate 
rating fortis result 
Р 
айлын This в a broad query for an 


actress. Different users may be 


User Location: Portland, looking for different things. 


Oragon 





The result contains a variety of 
User intent: Find ЖШС, elel information such as 


io infrmation images, brief biography, recent 
[рте | ics ‘Movies, and an option to see 





(ез ima -a more. Moderately Meets to 
ETEN pewsleelebriy Highly Meets is an appropriate 
gossip about Emily Blunt m rating for is result. 








This paga provides the 
requested lyrics on Coldplay's 
опса website, Users can 








— ECT] 
lyrics] all albums the song appears 
User Location: Bilings, | sse ааа ied 
шш? و‎ 2-8 
User Intent: Find the ulia ^N pesce 
ees ta isi жие» Additionally, users can trust 
nom 
К ОЗ 
pc Eurer 


ofthe source, which makes 
this result very helpful for 
many or mast users. 
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‘Query and User Intent | Result Rating Highly Meets Explanation. 





“The LP for this web result, 
contains authoritative infermation 
Query: [1969 moon about the Apollo 1t moon landing 
landing} оп the officiai website of NASA, 
em ыы ‘which was the organization that 
User Location: Naw iL ‘ovareaw thie mission in 1969, 


Haven, Connecticut Кн = Г وم‎ ‘The MC has a lot of helpful 








zm information and content, 
User Intent: Find accurat Including a detailed summary, 
and authoritative video footage, and images of the 





information about this тооп landing that users can 

historical event. Мем and explora. This result 
wouid be very helplul for many or 
most users, 





Tris в an informational query on 
a tariy broad topic. Diferent 














сому: еп education nae may ba ig for rant 
ange yel accuracy and 
user Location: Memphis, | саа здесна eben udi of ihe source are 
ре Er cn eprint 
User intent: Fina apr sam manna | m RT s A 
internaten or recent nava | Тете one on pubis on арза newe 
{bout educaton in STEM бершш: Tre МС af the aree 
ids Са, Senco, coves асат fire reform 
Technol Engineering STEM education nthe US. and 
nd maremas) incon cata fom expat 
oues. Tha еш! would be 
Pep for many cr most users. 
[Query mento and enn 
pi mia LP i ona кено 
fact checking ana datu 
[uter Location: Las ждө. The page cats tie 
Vegas, Nevada mmm Sry False, hen provider а 
x n — Li 1 | credible explanation of how the 
iier hagni Fi saute | usin roti Ix > author came o fat солоп, 
omaton about а апо | „ДШН от ташда istry of sir мап 
ara mur of Mentos egenis. Та uma would be 
and Cone Cua can cause vay hd far users seeking 1o 
a which am apre Fel a edes of animo. 
nth temet in 2006. 
Query pi een 
оа Tris LP is an opinion piece 


ritan by the Editorial Board of 


usce Mecca P ‘he largest newspaper in the U.S. 
state of Minnesota. The MC of. 


the article expresses 
T^" | walbinformed opinions about 


Minneapolis, Minnesota 


User intent: Find 
information or recent news 
эмал PA Fleck, who һай | 29207 nire itch 





uL EE flecks upcoming tenure as head 
(fant масава a пам hort ouch апа оша ву boo. 
eraba ae teres lo many or most users 


few days before this when this query was issue. 


example was written. 




















Copyright 2021 no 











‘Query and User Intent | Result Rating Highly Meets Explanation. 

‘Query: [batman logo. 

movie review] 

User Location: Orlando, This LP is an in-depth review of 

Florida the queried movie by а fim etic 
on fata reputable newspaper. This 


User Intent: Find reviews 
‘ofthe 2017 fim The LEGO 
‘Batman Movie on credible 
news or entertainment 

















result would be very helpful for 
many or most users. 
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A rating of Moderately Meets is assigned to results that are helpful and satisfying for many users or very satisfying for 


Moderately Meets results have fewer valuable attributes than Highly Meets results. Moderately Meats results should 
Still fi" the query, but they might be less comprehensive, less up-to-date, come from a lass authoritative source, etc. 


Moderately Meets results are not low quality, out-of-date, or inaccurate. Moderately Meets results are generally 


average to good. 


13.4.1 Examples of Moderately Meets (MM) Result Blocks 




































































Moderately Meets 
Query and User Intent | Result Rating к 
The LP for this web result isa 
Query: chute) highs quaity page on 
crunchbass com. a member ste 
User Location: Boise, | she) спелае n the TechCrunch 
idaho шеле network vith a proie of 
men, [ut ow COR TO aa | Shutterfly. The MC has a tot of 
User intent: Go to the v " helpful and accurate information 
Shuterywebste ord | “=== about the company and в easy 
Information about the fo read on a mobie device. This 
company теги! would be hou for many 
Users or very һер for some. 
The LP tor tris web result 
contains information about Tom 
жеке Cruise, which would be helpful 
for many user or very пары for 
User Location: Tom crie Pence, biography. some users. The photos and 
== pian in maln inks сап easly be 
س | ا‎ e ege 
User intent: Find orm ast i elena 
ای ب ر ی‎ Note that there are many, many 
N pages about Tom Cruise and this 
gest about Tn см. тегш в not пері enough for a 
‘ating of Highly Meets. 
Quay: ааа, This в a broad query for a city 
аары Different users may be looking 
User Location: Denver, арнын 
(Colorado PTL, [me rest shows a map of 
€— Seats Ed 
Infomation, news, maps, ea has санаа, 
Espero oom | 3 Sete тор. Tns result mey b 
Washington. e 
3 3 possible user intents, as the 
User Location: а 
етен ИЕ е Query is аво the name ofa 
Су ot Angus перета " movie, song, musical, and more. 
Эше Sot ۳ A However. sica the user location 
User ott Find А noh scat та теш 
мн d would be very help for some 
жо, aono, munion, sers -pecllcali users who 
толоо, or some other 
periret t 2. are looking for the schol. 
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Moderately Meets 



































Query and Usor Intent | Result Rating ses 
na This is a result to download one 
fe YouTube apps Mable 
[Query urbe) here want answers and results 
hi Locas: enii Fh away and тау not be able 
ишп وي‎ rir. 
E 7 Sa | onion and insialing apps. 
A "poste Ihat users want ene 
is ett Gott эне YouTube apos, bute 
probet not e dominant nent 
for том мат on the go. 
фк haan This result displays the name of 
тоел озара стык Sone 
шег тау have a ned met 
User Location: Miami, cs 
fous Nicaraguan córdoba | g Bani te rame the 
st ЧГ” [trong but many users may 
‘User Intent: Find out what ` ea асите агалай 
Minarapa, отеу и. helpful for many users or very 
Ead what К cca ika, ea en ише 
те exchange rate, et 
ашу: here ta медна | [vrama s tarsam y amant 
in the us) эт Waringin. DC, e rert The description in this resut 
[stara и д ый бн block may be helpful for users 
User Location: Louisville, | Зе sat By Noh Coran Who are alroady familiar with this 
kenu Кашан томны | Fj | [атат теш States. Many 
Re ron users would probably 
luser intent: Find where | rma iene nee жап sera тар ora carer 
фе ише ct VO i encre 
кемеа he 05 : 
Query lovey reat you 
fakes The LP fortis rcs website 
SELIG -every rea You Toke pene ha the Tace tee. 
User Location: Fort je iind Thara aro many lyrics websitos. 
Davis, Tes reme. |, E M nouis on tna web and many page are 
ШОШЫП Du ЖГ» at 100% acra. Moderately 
м Иа Мена ean appropriate ratna or 
ics to tne sona “Every erage pages win he 
an vou tae” wich requested yc, 
was writen by Sing. 
obey: кайаш Kristen Wig! The Offeisl Website of | This is Kristen Wiig's official 
pie мавы bul he олу contents 
er Location: aw Я 
sar Location: new Yor, | ЭСЕ iria to her talent and PR 
dn En mere yg agencies. along with a note that 


User Intent: Find more 
information about the 
‘actress, comedian, writer, 
and producer 














says she в not on any social 
networking sites. While this 
information is helpful, тапу 
users would want to see more, 
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13.5 Slightly Meets (SM) 


A rating of Slightly Meets is assigned to results that are helpful and satisfying for some or few users. Slightly Meets 
results may serve a minor interpretation, be low quality, have stale or outdated information, have some minor 
inaccuracies, be too specific, too broad, too niche for the general user population considering the query, etc. to receive a 
higher гайга. 





Important note: On mobile phones, the "cost of a click" is high. Many users decide which result to click or tap on based 
оп the title of the web result on the search results page. (Note that these tiles usually come from webpages.) A result 
with a very misleading or exaggerated title should be rated Slightly Meets or lower, due to the poor user experience that. 
occurs when the landing page does not match the expectation of the user when clicking or tapping on the result. 


13.5.1 Examples of Slightly Meets (SM) Result Blocks 





‘Query and User Intent | Result Rating Slightly Meets Explanation 





‘Query: [briney spears] 


‘The LP of this mobie-tiandiy 


[User Location: Baltimore, | rane spears a fr ноки! web result has a 2006 arce 











Mente LZ ——— ‘bout Britney Spears fling for 
ee marre ida D wa) "^ | divorce. This is very old, stale 

User Intent: Find were 

erint Pd ا‎ таша evs making the fest 

‘Spears (eg. current 

news, pictures). 

Query: honda odyssey] ‘Although the LP has 

comprehensive information from 

User Location: Miami, | 2010 Honda Odyssey -Kly Bue a reputable source, and is easy 

Рота. Book fo use on a mobile device, the 
аатина, А information is about he 2010 

User Intent: We wil = 2 





7^" | Honda Odyssey. This 
information would be considered 
s sale for he query today, making 


assume that users are mene 


Interested in the currant 











Honda Odyssey model, "his result helpful for some or few 
Unless specified otherwise. шет. 
The LP ofthis web result is about 
Query: о! dog] фе movie "Hot Dog,” which came 
out in 1984 and is a minor 
User Location: ius Tosi € interpretation of this query. 
Sunnyvale, Сайота | Sevis canann Because this is an unlikely 


User Intent: Find ањ mitur 





Zl, intarprotation, this web result 
would be helpful for few users, 


information about hot ‘even though tha landing page 
dogs, such as recipes or has good content from а 
палот information reputable source, and is easy to 


ше on a mobile device, 





“This result provides contact 
Information for one of ihe twa 
‘motorcycle shops in Potsdam, 


Query: motorcycles} 








User Location: Potsdam, New York. It is possible this 
New York a . " would be helpful for soma users, 
s [vw Wow but many users would be looking 
User Intent: Find for information. Additionally, 
information about Ses жеге is distracting 
motorcycles X information (a isting for an 














insurance agent). 
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‘Query and User Intent | Result Rating Slightly Meets Explanation 





‘Query: [abe lincoin's This в a Wikipedia page that has 

















Lato restr ote una hae tor ш шз presents, 
Б Кы каш Анат La 
асосан | س‎ tnt rt 
sr Loca umen | oem pranier played ai 
Ko тотема аай |= EE Rous that the anemer le 
User Intent: Find this. heb a павлин tor ét тее за pas nene د‎ € 
prada po page to find the answer, making 
Etta tts an pooper 
‘Query ack of sex and Te quatty aturing in ts 
Бы е esta bya 
Rag Lian at ن‎ es uou empate 
RE EN ЕЕ 
usar Location: waco, =, Mae! ae ea ough 
Tas A fre o's aout he qun e 
057 pave ow quay a 
er intent: Fins hak tor Eg T oud be ары 
arta еш [ен 





Query: [nsi spa] Burke Willams а day spa that 


User Location: San Jose, offers many treatments Ike 

















ioma : zr, massage еру. skin care, etc. 
Caio en | to dote ito hal сше 
NOR вета tie rota defiled nail 
ia in or near San Jose. " 
lis D 
‘Query: what ype of 
sharks live in rere] 
ere vote “This result block does not 
User Location: Tucson, | Зее ано ВЕ contain enough information to be 
Arzona” Eee ri Be, fy Sats hing and mentions only 
ЕЕЕ a SO ete |one type of hark. Mast users 


User intent Find wouid need to do further 























as | munt 
SS re e ala San Zl, [the information. 

кайан eina ad have to expand or click on it to 

|code for phone numbers. en 





ses the details, s0 1 is not that 
теры. 
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‘Query and User Intent 


Rating. 


Slightly Meets Explanation 





Query: i] 


User Location: Rock 
Maryland 





User Intent: Go to the 
IBM website or find 
information about he 
company 





This block contains images of the 
logo for IBM, but these images 
are not particularty helpful for this 
ату. 





Query: леге is nibini] 


User Location: Oklahoma 
Ciy, Oklahoma 


User Intent: Find the 
Jocation of "Nibiru", which 
‘was a ional planet in 
the movie Star Trek Into 
Darkness, as well ава 
pseudosciantiic outer 
planet in the solar system 
hat doesnt actualy exist 





Pianot tiru nes оревите 
iy ane DOOMSAY a on Pe woy 


Tara: сї Аи wi led 











The ше of this article makes it 
appear as though the arice 
‘describes an impending 
catastrophe. Yol, the MC states 
in its third paragraph that Nibiru 
“doesnt actually exist" and cals 
believers in i "conspiracy 
жеопыв^. The title contradicts 
the content of the article, and as 
a result could mislead users who 
Sont read the ful rice. 


Note: The "eost of a click" is high 
on mobile phones. Results wih 
very misleading or exaggerated 
Mes shouldbe rated Slightly 
Meets or lower. 





Copyright 2021 


ne 





436 Fails to Meet (Falls) 


A rating of Fails to Meet should be assigned to results that are helpful and satisfying for no or very few users. Falls to 
Meet results are unrelated to the query, factually incorrect (please check for factual accuracy of answers), and/or all or 
almost ай users would want to see additional results. These results completely fail to meet the user intent, such as a lack 
of attention to an aspect of the query (or user location) that is important for satisfying user intent. Fails to Meet may also 
be used for results that have very stale or outdated information, be nearly impossible to use on а mobile device, etc. 





‘Search results should never surprise people with unpleasant, upsetting, offensive, or disturbing content. For this reason, 
all of the following types of content should be rated Falls to Meet if itis clear that the user is not looking for such content: 


Harmful to Self or Other Individuals (Section 7.1) 
Harmful to Specified Groups (Section 7 2) 
Harmfully Misleading Information (Section 7.3) 
Untrustworthy (Section 7.4) 

‘Spammy (Section 7.5) 

Porn (Secllon 15.1) 


13.6.1 Examples of Fails to Meet (FallsM) Result Blocks 





Query and User Intent | Result Rating Fails to Meet Explanation 





‘Query: [dogs] 

This is a broad informational 
query foc an animal, Different 
users may be looking for different 
things. 


User Location: Ptsfild 
Charter Township. 
Michigan 


User Intent: Find ıı [Tis result block shows 
information (e.g., pictures. Pío Ho» vistcin-nerson information for 
breeds, training details, three diferent dog cara providers 
etc.) elated to dogs. near the user location. However, 
- the query is very broad and itis 
unlikely users want lo go 
anywhere in person. 





Note: For this query, itis 
vary unlikely users want to 
do anywhere in persan. 








Query: Ike] This в a broad query and itis 


unclear exactly what he user is 


User Location: Eust, lacking or. 


Florida 





‘This result block shows 
visti person information fortwo 
businesses that contain the 
name Mike. However, the query 
" i very broad and itis vary 
unlikely these businesses are 
what users seek given the query. 


User Intent: Find 
information about 
‘something related to Mike. 





Note: For this query, itis 
unlikely users want lo go 
anywhere in person. 
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‘Query and User Intent | Result Rating Fails to Meet Explanation 





Query: Патра rays] 


This result block shows 
information to visita sports card 
store in person, which happens 
о nave the baseball team in the 
name. However, remember that 
for tris query, tis unlikely users 
Se | want to go anywhere in person. 


User Location: Tampa, 
Florida 


User intent Find 
information (e.g, score, 
roster, recent news) about 
а professional baseball 
team in Florida called 
Tampa Bay Rays. Note: This business was 

fabricated forthe purposes of 
this example, and dos not exist 
in eal ite 





Note: For this quar tis | с 
unlikely users want ia go 
anywhere in person 








WRAL is a television news 
station in North Carolina, Users 
want to go to the website or find 


Query: а] Information about the station. 


User Location: Wake 


County, North Carolina in this example, clicking on the 


blocks show information about 
the businesses corporate 
headquarters. Note that 
television stations (ike radio 
stations, newspapers, et. 
usually have special contact 


User Intent: Go to the 
WRAL website or find 
other information about the 
‘ews station in North 
Caroina. 





Ed 
1 
i 











тнт m ‘eration rhe publ tatis 
feront than tar corporate 
ee ies ادم چ‎ headquarter information, Aio, 
тнк wh the query just being he 
тате of the dan. ts very 
inkl hat users want nis kind 
a inormaton 
Query: eas fam SNO 
етше тив result block shows the 
>з Texas Farm corporate office, 
User Location: Dalias, эол в not перы given the 
Teras ‘ser тале users could not 
4 bed s EI ‘physically go to the company's 
User intent Fina Corporate ica to buy or ват 
петао about or | „ cem Stout the product, Ferstearim 
where to buy, tertizer fm | 7 757 the kind of producat you 
he company Texas Farm” | 9 ice ould have to buy ata sore 
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Query and User Intent 


Result 


Fails to Meet Explanation 





Query: tunited 656] 


User Location: Skokie, 
Minois 


User Intent: Find 
information about fight 
|656 operated by Uned 
Aitines (eg. whether the 
ight is on ime, what gale 
tis departing from). 

















т ~ 
mint ү 
mu ~ 





This result shows information or. 
‘church, an evant venue, and 
tha closest airport. However, 
users clearly want to know 
details about a specific fight, and 
"hero is absolutely no information. 
in the block about the fight. This 
result Fails to Meet the user 
intent. 





‘Query: [german cars] 


User Location: Toledo, 
Ohio 


Subaru of Americar Subaru cars, 
Seans, Sv Crossovers Wagons 


m 
a 
В! 
af 


This LP is the homepage at 
Subaru, a Japanese car 
company, not a German саг 
company. 



































tsar iment Fed ppl nm ЕР 
information about German |. fiam cas senes wisgo. (The pati concitat Ранне 
vans Gams - estis uso int ond кона 
homepage ofa German Bo help far no or very few. 
automaker. үк 
амт Romany toast This LP is the homepage ога 
E ere rient 
sao US. var woul aed 
Cone Pee contol Mie 
nr Loca: К SS company as d 
penaa ж MM. [Pie ptio Dre à 
een iude. Ж TET TET OE TN | ce cipe e vad 
КЕШЕСЕ фе пеше amate те ы 
urere E Е 
aono omoa completely Falls to Meet tne 
ease a pos user intent. 
vary: [aring is 
ВЕЕ 
though is a sh 
User Location: Chicago. | ц мыл. ae website for information about 
Weis Nev toto st does net 
iudei dindal ET contain the information 
User iments ind the | memet rn oues by Rmus The 
name of the starting nmm ‘result completely Fails to Meet 
атаеы or ha New re wer mon 
Senseo erin 
К 
[query iata to 
уйше тайа р] | scutis алдый. 
æ as honepags стената 
ee Lec: Dua rou чайла Pa completly laeto 


User Intent: Navigate to 
Yellowstone National Park 
от the user's location. 

















respond to the speci Device 
Action query: to navigate to the 
park. 
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Query and User Intent 


Result Rating 





Fails to Meet Explanation 





Query: [doctor salary] 


User Location: San 
Francisco. Самота, 


User Intent: Find 
information about doctor 
Salaries. 


үт 2019-2074, annual tution and 
fees а ute medea senasis 
sevrage? approdimataly $31,783 for 
Stata residans and 355,204 or ron- 
Тиде At private schol, ton 
n fees averaged $52,093 for 
‘sides апа 350.476 or non- 
resident stent, 

How Dal. Pay freien Sena 


Served Steiner 








The answer in this block is about 
the cost of education, nat salary, 

Which в misleading and doesn't 

answer ihe user's query. 





Query: [sony child 
development centar west 
point phone number] 


User Location: West 
Point, New York 


User Intent: Find the 
telephone number for tha 

‘Stony Child Development 

Centerin West Pont, New 
York 


(845) 446-5504 





The answer in this block provides 
tha phone number for the West 
Point Club, not the Stony Chie 
Development Center located in 
Wast Point, New York. This 
тези! Fails to Meet the user 
intent. 





Query: wno invented 
stars] 


User Location: Lubbock, 
Texas 


User intent: Find out 
about the origin of stairs. 


“stale have become such а commonplace 
fiture in contemporary architecture that itis 
easy to forget that they were not invented 
until 1948, by Swiss architect Werner 
Básendirer" +s 2507 


ty ol me sta legen 
нча. 





The answer provided in this 
result block s factually 
inaccurate. Stairs have been a 


соттоп architectura feature of 





1948, and they were nol invented 
by a parson named Wemer 
Bösendörte 








Query: [young famous 
app developers] 


User Location: Lubbock, 
Teras 


User Intent: Find 
information about, or a ist 
ot, wol-knawn app 
developers who are 
young. 














This result includes several apps. 
te download, with an option to 
зев more apps. The user is 
looking for information about app 
developers that are wol-known, 
Sot Fails to Meet Ine user 
inten. 


Note: Unless you fel tha user 
intent lo download an app is 
strong, results that allow users to 
‘download apps are generally 
less helpful. Mobile users want 
answers and results ght away, 
lat moment. and may not be 
abia to spend a t of ime 
downloading and instaling apps. 
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‘Query and User Intent | Result Rating Fails to Meot Explanation 





Query: [send text 
message to john that im 























к 
и ачаа The user clearly wants to send a 
Bst Location: Haw iet tex message. The Device. 
isi мна ج‎ Action result to "Send an email" 
User intent: Send a text | Recipient: John өм conci Fil 
message fo а contact 
famed ohn vat the | Body fm ning lale 
‘message “Tm running 
ше 
Query: [weather] The user clearly wants weather 
User Location: Query Time: 2013-4-23 6:00:12 ‘formate, The Device Action 
Mountain View Calfomia r 
Acto: atalar E inappropriate (and even absurd) 
bis ted nd ae: ior fe тоттайоп query The 
Я ‘oul completly Fatis to Meet 
‘weather informasion fr Alarm Time: 224300 aa ce 
Mountain View, Calor 
(Query атап] 
ttis extremely unikely 
User Location: Anaheim, | (polenta а no chance 
Catonis ierretsion) tat e quay 
Pian па "co for information ona cy 
User ntent: Find P e n ia in Turkey ced Batman, ver 
information about he -E that the user is ated in he 
Тайпа superhero hat DS United Site. No or almost по 
appears American — mere would bo saec with is 
comic books, moves, and mat 
‘Shion show 
P i itis extremely unlikely 
(Query ата] Gum {potently поема 
EM ierretson) tatis query is 
tsar Location: Aun, | ы looking for information on а ety 
nl E în Japan caled Obama, given 





"hat he user is located in he 
United States. No or almost na 
usere would be satisfied wih this 
result 





User Intent: Find 
information about Barack. ا‎ mama 
Osama. 














‘Query: weather paris, 


эн ^55" “This result completely ais to 
кылы a 55 жауа жеме мы 


Teras . к— find weather information for 
Paris. Texas, and not Paris, 


France. No users would Бе 
User Intent: Find current ” Г Меле wih tis ros. 


‘weather information for 
Paris, Texas ess 
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Query and User Intent | Result Rating Falls to Meet Explanation 
Query: tunat is е tra 
quarter eamings torge | 27.02 01000704) 
و ا‎ аа Tis result does not answer the 
س ا‎ i А user's question to show hid 
кт quarter earings fr GE. No 
PER Stes wel d win is 
тов recent йы | не н 
Earnings omation fr 
General Esc 
тив resul rci one of several 
TS ae posetie user mienie. as the 
ii gal f шу is aiso ha name of a 
Query: ey of angel e ‘ove, song, musical, and more. 
eran! دنا‎ Since бе user locaton isin 
90. linca City of Angels Independent Chicago vay tew ar no users 
ынан Slides болоо ке; Ms ноша be interested in seeing 
есе A | P information about а school by 
: "hie name in Los Angelas ais 
ой an E то wel known ouise ot ina 
пате, araon ETET ly. Ив very unikoly hal people 
y IRAR tide of Los Angeles woud 
шша thie quay to searen for this 
schoo 
Tris result block shows а go rart 
Query: co kart ar satel arcade in Чате 
Pennsywvania, and o other go 
User Location: кап arcades in the surrounding 
Warrington: Pennsylvania سی‎ are. These arcades ae paces 
AIT XCTI. io go ide go karts, not purchase. 
User intent: Find go karts tham, soe resul completely 
for sale in te Waringan- Falls to Meet he user ent No 
Pennsylvania ara. m sere would be solid wih tis 
тшш 
(Query: ары 
User Location: San Ralphs is а nationwide 
Clement, Caitoma Supermarket cha. The 
localons are alin San Diego, 
User Intent: There are lT |Calfomia, which is a major city 
wo posible user intents: € South ofthe ser location (about 
most users probably want s x {0 mies away). These res 
to vist a nearby Ralphs are loo far lo be hel. 
Iocan ar go 10 he = 
webste. mmm. 
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Query and User Intent | Result Rating Fails to Meat Explanation 
User issuing tis query want io 
poto бе Amazon wanene. 
having intonation about har 

(Query [amazon] a iat or call ne сирок опсо 
{eran onina company ou nat 

[uter Location: Ausin, be heit. Ve very few people 

Texas К экы Шы [ever go to an Amazon corporate 
ce une nope ee 

ser Intent: Go o the pubic. H someone need o ро 

Amazon webera (о peate for аптадан ог 
sens vist thy коша need 
gei that inmallon iom 
отоо al бе company or 

— enki una a mor space quan 
сому: [etatis he coena 

Өм Nox or large ety 

К ал" A пана эми is inappropriato 

User Location: муе : 

ltr cation а i sss. [and completely итер is 

Er MEE ET RT TET esl nny no use waldo 
leitet nte | ganna a aic wih res 
oes! largo ciy to My 

Beach, South Carina 

[Query [ausralan open 

ane ines res 200] Tris LP is about he 2004 

——— шарап Орап not ha 2008 
[User Location: Kent, | tyes ie Ашап Ороп: does not pay 
рене Ке ИА foto [ation to an aspect йе 

Ө db Se (uer that important or 

User intent: Find a page | ines isting usr ar e 2008) 

That daplays не дов | Ser = Sewn mans The resul Fall to Meat ine user 

rera inge ret or ч 

irri tmamnt. 

Query Ioh loss fe 

а This LP has information about 

Я atey mania ran ой ies p fen ana has he 

[User Location: Denton, | Chap 16 Pier ana Miiminnow words “ve years oid" on the 

SS | Банини page. This result Fails to Meet 
Mem русс. Fuse intant because has 
LAND мука neiches oniy and odd 

Formaten abou tooth ко than oni 

Jasa na fue year cid helpful for no users. 

E 

сому: whats wang in 

spanish] me translation in the result block 
к comnis wrong and 

User Location: Denton, mafiana " " hein ecl 

Toas جج‎ > 

User intent: Fina how to bat падпа vimos! not 

say the word "wedding" in ы 

ЕЕ 
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Query and User intent [Result Rating Fails to Meet Explanation 
TTT 

ImetPisatunous bip poet 
шшш cut wie aig har and 
Me Ө ы Russ | pyc The page meniions 
User intent: Gotothe | 3s йити, |l жш y ON raw | American Express," butis 


American Express card 
‘website or gat information 
‘bout the company and йв 
products and services. 


Insufficient related lo the query 
to be helpful lor users, so it Fails 
o Meet the user intent 





Query: [ow to quit 


























тока This LP has gibberish text. Read 
то ре воен. -ws у his sentence: How do you make 
User Location: Denton, | 007277907" we а cigarette symbol on the 
= TS з | ram ma ui муюн ee The quality of he 
кел landing page в во low thatthe 
User intent: Find f page Falls to Meet the user 
information on ways to quit ent 
smoking 
‘Query: american beauty] This i the officiai website of the 
movie "American Beauty. 
User Location: Mountain | American Beaty: Dri However, he landing page is 
| View. California LAN ACUTE, extremely dificult to use (even 
Trev tne seems broken on a mobile 
User intent: Learn more hono) and there is no slistying 
about the movie called Br helpu! content on the раде t 
“American Beauty Fails to Meet he user intent. 
Query: [starting order for — 
today's nascar race] 
mmm EJ ‘This result shows a NASCAR 
мнн Darian, үст E schedule, bu does not have the 
i Staring order. The content isnot 
S IM oar d nat th uter ia looking or an 
User intent: Find LE 
бынын. Fails to Meet he user intent. 
staring order ol the 
Nascar race at day | EET 








Query: fair canada phone 
number] 


User Location: Seattle, 
washington 


User Intent: Find the 
customer service phone 
number for ho airine Air 
Canada. 





1 (888) 247-2200 











Remember that you should 
heck forthe factual accuracy of 
answers. For this result, go lo 
the company's page at 

ow aircanada cemlenleustomer 
(сапы and select USA to check 
the number shown here. 
According lo the aiine's website, 
‘ha correct phone number is 
1-888-247-2262. Because this 
answer is incorrect, ¢ should be 
‘ated Fails to Meet. 
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‘Query and User Intent 


Result Rating 


Fails to Meet Explanation 





‘Query: [obama news] 


User Location: Norfolk, 
Virginia 


User Intent: Find recent 
news about Barack. 


Obama, the 4ath president 
ofthe United States 


Obama lunes Piane Far ATi 


И ‚ 
WalenüC-Hedémaek |» жо м 


‘The MC of this page is an 
example of a demonstrably 
inaccurate news raport, including 
a fictious quotation by Barack 
Obama. Because the content is 
presented as a factual news 
article. itis inaccurate with an 
intent io deceive users and 
Should be rated Fails to Meet. 





‘Query: [does smoking 
cause cancer] 


User Location: 





User intent: Find out 
whether smoking causes 


Smoking des MOT cause ung 
ance in feo might protect 
Firewall. mee | p 
a | db d 
Tar 10,1999 те йолга acri 

ie takan fem Laron A. Coby book 
In Defense of smokers 


Despite strong medical 
consensus linking tobacco to 
ung cancor which can be found 
at authoritative sources by 
searching lung cancer and 
Smoking] he LP claims there is 
Po connection between smoking 
and cancer by selectively 
interpreting some experiments 
done on animals. 


Additonal, this is а YMYL query 
оп а medical topic, so accurate 
and trustworthy information is 
highly important. This page 
directly contradicts 
welLestablished medical 
consensus and may potentially 
cause harm to users, so it should 
be rated Fails to Meet. 








Query: [betty white] 


User Location: Lexington, 
Kentucky 


User Intent: Find 
information about he 
American actress Belly 
Whita, such as 
biographical into, 
fimography, awards, 
current news, ete. 














Betty White was alive at the time 
this example was written. This 
LP i a social media profile page 
created by an unknown author 
"hat was designed о look ko а 
memorial page for Већу White 
and falsely states that she died. 
‘This page в factually inaccurate 
and deceptive to users, so it 
should be rated Falls to Meet. 


Remember: For informational 
gueriesiresults, itis important lo 
Chack other reputable sources to 
‘termine ithe result i factually 
accurate. А quick oniine search 
Tor [ену whita] would айон you 
о confrm that she was st aive. 
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Query and User Intent | Result Rating Fails to Meet Explanation 
ey Cyrus was аме at tne ime 
ths example was writan. Tis 
LP is an arto itn by an 
unknown author that was 
osigned o ook Ike a news 
Story and tao states hal she 
Ged. Furthermore, tne page в 
found on a ow quality website 

(Query: адыны and he aricie hand bean 

SRI pite, cad, or removed 
ae proven taise. Ts page is 
factual inaccurate and 
ото мош he Р; "lu | deceptive ta sors so it should 

American singerlactress NTR 

Мсн او‎ Remember: Just because a 

а Мерз has пе look and ев ol‏ ی 

decographimography atonal news site does NOT 
"ean that you should consider 
{he reporting о ba facta 
accurate wihout fure 
research. Walch out or etes 
that present hemseves as 
meus but ware actualy created 
ith the intanto mea or 
enor users 
This query в seeking historical 
intoraton The LP of tha web 
results on a welknown whe 
supremacist hala ste read more 

‘Query: poiocaust history] | толаш shyt bat Bete or nars). The discussion on 

pre ‘his page s accurato and 

User Location: porc misleading. since the Holocaust 

Cooperstown, New York | Uem erem daten جيم‎ в ап extremely we document 
torial vent 

User intent: Fina 

historical Information about Remember: Pages nat arecty 

ha Holocaust. conrade! welresabished 
‘storia facts shoud be rated 
Fails to Meet, unless the query 
eariy cas бе шег 
seeking an alternative viewpoint. 
This LP is a biog post presenting 
la actualy unsupported 
onspiracy theory inat Angela 

nn hia bilai Merkel is the daughter of Adolph 

Span. Hiller. Because the MC is 

eene Mone ianari 

ser Location: Кс Сатри Рай to Matha 

Соемиас сете азе] user intent, even though the topic 

User Intent: Find the | nk 2005 ar es ofthe page matches the query. 

breit nn a Nate: A more helpful result for 

‘daughter, if Hitler in fact Абау тон СИ. 

ct ‘Adal ilr ad по chldren, or 
Provide formation on the novel 
Hier's Daughter by Jackie 
Franc 
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ESSE aes EE EET 
SEE 
ENIM 
(Query: Balam Proof that Istam is Evil, Violent, and. was created for the purpose of 
meni pean eao nagam t, 
er tnt: tegen | rms RE iem 
Dres Eu ee m | pantera To 


User intent Find 
information about the 
religion of islam. 








tle tt sae la ina reque. 
Fah lte orta tinere 








‘author lacks expertise in the 
oic: the writing has frequent 
misspelings, typos, and 
grammar errors: and the page 
Seems ia exist to promote 
intolerance or hate. 
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14.0 The Relationship between Page Quality and Needs Met 


The Needs Met rating is based on both the query and the result. You must carefully think about the query and user intent. 
when assigning a Needs Met rating. 





Tho Page Quality rating slider does nat depend on the query. Do not think about the query when assigning a Page 
Quality rating to the LP. 
ley 


WA Loves Nedium Highest 


‘Some results don't have a Page Quality slider. If a result block has no Page Quality rating slider, you do not have to give 
а Page Quality rating. If there is a Page Quality slider, please assign a Page Quality rating based on the landing page. 


Here is some guidance about assigning Needs Met and Page Quality ratings 


«Useless results should always be rated РайвМ, even if the landing page has a high Page Quality rating. Useless 
is useless. 


= On-topic, helpful, but low Page Quality results should get lower Needs Met ratings than on-topic, helpful, and high 
Page Quality results. The Needs Met scale encompasses all aspects of “helpfulness,” and many users find low 
Page Quality results less helpful than high Page Quality results. Your ratings should reflect this. 


‘© The HM rating should be given to helpful, high Page Quality pages that are a good fit for the query. The HM 
rating may also be used for results that are very helpful, medium quality, and have other very desirable 
characteristics, such as very recent information. 


* The HM rating may not be appropriate if a page has low Page Quality or has any other undesirable. 
characteristics, such as outdated or inaccurate information, or if it is a poor fit for the query. We have very high 
standards for the HM rating. 


*  SMisoflen an appropriate rating for low quality but on-topic pages. However, a page can have such low Page 
Quality that itis useless for nearly ай queries. Gibberish pages are a good example of pages with low Page 
Quality that should be rated FallsM. An exception to this is queries with clear website intent, where the target 
website should be rated FullyM even ifthe page has low Page Quality. 


‘© Remember that if a page lacks a beneficial purpose, it should always be rated Lowest Page Quality - regardless 


of the page's Needs Met rating or how well-designed the page may be. Please review Section 7.0 for a summary 
of other types of Lowest Page Quality pages. 
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‘Query and User Intent. 


Result Block and LP, Needs Met Rating, Page Quality Rating 


Explanation 





‘Query: [how do electric 
cars work] 








Electric Car Motors - How Do Electric 
Car Motor... Ezine Articles 
nents soni. Moles Motors. 

ete елау 0 10 2008 This engine which 
T on goce ipl has donated e 





Needs Met: This pago would be 
helpful fr few users. 


Page Quality: Read this contant 
out loud. The article onthe LP is 
‘hallow and unintentionally 
humorous. The MC has four 
paragraphs of text, but t basically 
says that electric cars use 
‘electricity and have to be 




















Te 
User Location: Danas, sk du ETE. [ee 5 
Us 9 
User Intent: Find Mobility scooter - Wikipedia, the free Needs Met: The query asks how 
реу ИШ „ы cae tos ишк биде tae 
оско onta work. en m wikipedia arg/wiki/Mobility. scooter ‘on the LP is about mobility 
errori туго ИВ ЕП 
[ао sem pom apud 
‘wom de er a Page бойу: There tea itor 
TORUM ne шайр pege 
en ipsom To page fas 
CES Sonat Medium Highs 
к-ту. rn Шеше; 
Symptoms of Dehydration Needs Mat: Eventhough the 
bral id rice topical ha fomaton в 


Query: [symptoms of. 
dehydration) 


User Location: 
Memphis, Tennessee 


User Intent: Find 
information about he 
symptoms of 
dehydration. 


What are the symptoms of dehydration? Leam 
about the signs you can expect when dehydration 
begins and aleo ait 








‘not reliable and potentially 
misleading ог dangerous. 


Page Quality: This is а YMYL 
opi. This page has no evidence 
‘of medical expertise/authoriy and 
пой trustworthy. 








Dehydration Symptoms - Diseases and 
Conditions - Mayo Clinic. 
eros or. eye. 

oie та Saver engan a medien! 
Meroe an rame. etna B: Per 
"esses > npe in infants and сайт 
\тшыму and conto in мы Very dry mouth, 
slin anê тыс membranes" Ше гло, 
in - any urne вага diced wil be 
darker an nemo Sunken eye mor tme 











Needs Met: This result would be 
vary halpfuî for most users. 


Page Quality: This is a YMYL 
opi. This page is on a highly 
айлопавув medical website and 
has a lot of reliable and accurate 
МС. This page is very 
trustworthy. 
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‘Query and User Intent 


Result Block and LP, Needs Mot Rating, Page Quality Rating 


Explanation 








Query: [how many 
octavas оп а оа 


User Location: 
‘Cleveland, Ohio 


User Intent: Find out he 
amber of octaves on а, 
guitar. 





ШО 





seven octaves 











Needs Met: This block shows the 
number of octaves оп a piano, not 
a guitar. Even though this SCRE 

has a link 1o a high quay landing 
pago about pianos, is irrelevant 
fo the query. Useless results are 

useless. This гези! Falls to Meet 
the user intent. 


Page Quality: There is a lot of 
helpful MC on the landing page. 
The pago has medium to high 
EAT. Mediums to High is an 
appropriate rating. 
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15.0 Rating Porn, Foreign Language, and Did Not Load Results 


You will be asked to assign Porn, Foreign Language, and Did Not Load flags to result blocks when appropriate. Some 
rating tasks may also ask you to identify Upsetting-Offensive andlor Not-for-Everyone results. 


All flags should be assigned based on the result alone and do not depend on the query. Here is a screenshot of the flags: 
Pom No] Foreign Language. No | | Did Not Load. No 


Click on the flag name to select it. The flag will tum red and change the “Мо” to "Yes." For example, here is a result block 
that shows when the Foreign Language fiag should be used. 














(Query and User intent Result Block and LP, Needs Met Rating, Flag Explanation 
бшумен | EE отте Bantu webal 
rr bacon) The result а the target 
Locale: English (US) 8 л - FERRARA. ёт. Ен. Smp |page of the query, but is 
User Location: 3. BPAMEPWAHWENAE. а foreign language 
Anchorage, Alaska eet 
وم“‎ This result merits a 





User Intent: Go to the 


Baidu website. Fully rating and the 


Pom No | [Foreign Language Ves] | Did Not Load No Foreign Language flag. 

















154 Porn Flag 


Please assign the Porn Пад to ай pom pages, whether the query is pom-seeking or nol. The Porn flag does not depend 
on the query or user intent. 


This flag should be used if the content inside the result block, or the content on the LP, is pornographic, including pom 
images, links, text, pop-ups, and/or prominent pom ads. An image may be considered porn in one culture or country, but 
not another. Please use your judgment and knowledge of your locale, 


Remember: The Рот Пад is used to indicate that the result is рот. It doesn't matter whether the query "asks for por’. 


152 Needs Met Rating for Porn Results 


15.2.1 Needs Met Rating for Cl 





ir Non-Porn Intent Queries 
Ifthe user intent is clearly not porn-seeking, a landing page that has porn for its MC should be rated Fails to Meet. 


‘© When the user intent is clearly not рот, a porn result should be considered unhelpful or useless. 
‘© Uninvited pom is a very bad experience for many users. 


The following queries should be considered non-por intent queries: [iis], [wives], [mature women], [gay people], [people 
Missing), оу speedos], moms and sons], [pictures of gis, [pictures of women], [mers and daughters [cheerleaders] 
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Remember to assign the Porn flag for ай рот pages. Use your judgment and represent users in your locale. 





























[T пнен Landing Page mans [Porn Fag? 
evs Ente us) Fr oo har Eme scot pm. rat ves 
комы isa camen Engten us) End Pe живота ame naar "m 
esee вода (US) [кёре [iate н ves 











15.2.2 Needs Met Rating for Possible Porn Intent Queri 


‘Some queries have both non-porn and pom interpretations. For example, the following English (US) queries have both a 
non-porn and an erotic or pom interpretation: [breast], [sex]. We will сай these queries ‘possible porn intent” queries. 


For "possible pom intent” queries, please rate as if the non-pom interpretation were dominant, even though some or many 
users may be looking for рот. For example, please rate the English (US) query [breast] assuming a dominant health or 
anatomy information intent. 




















‘Query User intent Landing Page Rating Porn flag? 
Find anatomy or health 

breasts), English (US) Find anatomy ова [Example нм No 
Find anatomy or healh | Example 

нм. English (US) information about breasts. | Warning — this page is pom. [FM ы 




















15.2.3 Needs Met Rating for Clear Porn Intent Queri 


For very clear por intent queries, assign a rating to the porn landing page based on how helpful it is for the user. Even 
though there is pom intent, the page should sill be assigned a Porn Пад. 


Do not simply rate all pom pages for pom queries as MM or HM. Even though the query is porn and the result is pom, the 
page must fit the query and be helpful to get a high Needs Met rating. 


Pages that provide a poor user experience, such as pages that try to download malicious software, should also receive 
low ratings, even if they have some images appropriate for the query. 


























E E nang Poe кше О (Parag? 
arci 
ЭО ранно ва ронса ES cama [ran va 
егар | Warning — this page is pom. 
arci 
X il mahana an v 
ша | Warning — this page is pom. 
eem 
m——— RH енени " 
Finger? [ema 
ананна от rain us) [asamp T X-— À e 
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153 Reporting Illegal Images 
Child Pornography 


Note: When working on rating projects in any task location or locale, you must follow United States federal aw, which 
considers child pornography to be ilegal. 


Definition of Child Pornography 


An image is child pornography if it is a visual depiction of someone who appears to be a minor (.e.. under 18 years old) 
engaged in sexually explicit conduct (e.g., vaginal or anal intercourse, oral sex, bestiality or masturbation as well as 
lascivious depictions of the genitals), or sadistic or masochistic abuse. The image of sexually explicit conduct can involve 
а real child; a computer-generated, morphed, composite or otherwise altered image that appears to be a child (think of 
images that have been altered using Photoshop’); or an adult who appears to be a child; and the image can be 
non-photographic (e.g., drawings, cartoons, anime, paintings, or sculptures) so long as the subject is engaging in sexually 
explicit conduct that is obscene. If itis indistinguishable from child pornography, itis child pornography. 


Even if the pomographic image depicts children in a literary (think of the famous book "Lolia") political (think of political 
cartoons), artistic, or scientific context, please send the link to your empioyer/company as instructed below, 


Depiction of the genitals does not require the genitals to be uncovered. Thus, for example, a video of underage teenage 
Girls dancing erotically. with multiple close-up shots of their covered genitals, or images of children with opaque underwear 
that focus on the genitalia could be considered child pornography. 


An image of a naked child (e.g, in the bathtub or at а nudist colony) is not considered child pomography as long as the 
child is not engaging in sexually explicit conduct, or the focus is not on the child's genitalia. Visual depictions of adults who 
look like children (e.g., a 35-year-old man play-acting in diapers, or an obvious woman dressed as a schoo! giri are not 
child pomography. If you don't think its a minor, it probably isnt child pornography. However, if you cannot tel that the 
person in the image is over 18 (e.g., an under-developed 18-year-old whose body hair has been waxed), that is child 
pornography. 





Please report images containing child pornography (URLs only) as instructed by your empioyerícompany. 


484 Foreign Language Flag 
184.1 Using the Foreign Language Flag 
Please assign the Foreign Language fag when the language on the landing page is NOT one of the folowing 
* The task language 
£ langage that is commonly usod by arcani percentage ofthe population in the task location 


For example, most Catalan-speaking users in Spain also speak Spanish. Therefore, for rating tasks in Catalan (ES), the 
Foreign Language Пад shouid NOT be assigned to landing pages in Catalan, Spanish, or English. 
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Here is a screenshot of a result block that shows when the Foreign Language flag would be used, 
































Riel ГООО, 7 
ае m КЕГИ ре 
eed The result is the tar 
Locale: Engish (US) | piscem FERNAN, BT. Ык. Sp Mate ied 
User Location: Ж, ERWARTE їп a foreign language 
oce, [er 
Bata mataa е sa шум rating ana tbe 
er. Pomi No | [Fovegn Language: Yes] [Did Not Lond No Foreign Language flag. 
irent 


«^ Please assign the Foreign Language flag even if you personally understand the language, but most users in 
your locale do not. 

* Please assign the Foreign Language flag based on the language of the landing page, not the appearance of the 
result block. 

* Please remember to flag all foreign pages with the Foreign Language flag, even if most users in your locale. 
would expect or want a foreign language page for the query. 

‘© Sometimes it is difficult to determine what language the landing page is in. The LP may have multiple languages 
or no words at ай. In these cases, ry to represent users in your locale. Does it feel like a foreign language page? 
When in doubt, don't use the Foreign Language fag. 


15.4.2 Needs Met Rating for Foreign Language Results 


You must assign a Needs Met rating for all result blocks in your task, even if the result blocks have a foreign language 
landing page. 


In most cases, pages you flag as Foreign Language should be rated FailsM, because they cannot be understood by 
most users in your locale and are therefore useless. Remember that if users in your locale can read the language, then 
you shouldn't be using the Foreign Language flag. Occasionally, you will encounter helpful Foreign Language pages. 


Ifthe query clearly indicates that most users would expect or want a foreign language result, then the Needs Met rating of 
the foreign language page should not be FallsM. For example, please assign the FullyM rating and Foreign Language 
flag for baidu.com if the query is [baidu.com], English (US). 





Videos are often an example where foreign language pages are helpful and desired. Think about user intent and what 
pages are good for users. If the query “asks” for а foreign language song, band, fim, sporting event, etc., then a video of 
the song, band, fim, sporting event, ete. is helpful since it can probably be understood or enjoyed even though it is in a 
foreign language. For these types of queries, foreign language results are often expected. 


IF the video is someone talking about the song, band, fim, or event, the result probably cannot be understood or enjoyed 
and should be assigned а FailsM rating and the Foreign Language fiag. 


Note: If you are unable to evaluate the Page Quality rating of a Foreign Language result, you do not need to assign a 
Page Quality rating and can leave the slider at NIA. 
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Query and User Intent. 


Result Block and LP, Needs Met Rating 





Query: [dea of celine dion. 
singing sîl suffisait d'aimer] 


Locale: English (US) 


User Location: Lansing 
Michigan 


User Intent: Watch а video of 
Celine Dion singing this song. 


Celine Dion - S'il Suffisait D'aimer Live - 
YouTube 


ө YouTube app 








Ts video is just what English (US) users are 
looking fr, even though the video is not in 
English. The language of he LP is mostly 
English (even though tha video is in French), во 
the Foreign Language lag is not needed. 





(Query: [alex c] 
Locale: English (US) 

User Location: Davenport, low 
User Intent: Walch a video or 


listen ta a song by this German 
composer or find information 


Alex Cft Yass- Doktorspiele - YouTube 
© YouTube app 
pen on youtube com 





The query is for the German composer, Alex C. 
The landing page has a video with one of his 
songs sung by Y-ass (misspelled as "Yass" in 
the YouTube tile) in German. It would be 
helpful far some or many English (US) users 
who type the query. even though itis not т 
English. The language of the landing page isn't 
(аваг. Use your judgment when deciding 








E ج‎ ww eine iri Pope Lanka tag 
ery: asa aa an : 
bin П | — Кава Калай -YouTube ageyra ani Kanita n 
Locale: English (US) ep Filipino fim. The LP is a clip rom the movie in 
asar Filipino (Tagalog), the language spoken in the 
шш Ennen [US uae wh yn he ay man. 
up ena rng Un yo отан 
sor intent: ach tater ot = ишш Г Ту In 
апиын „тт E ER п, aree 


aboutit 
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15.5 Did Not Load Flag 
15.5.1 Using the Did Not Load Flag 
Did Not Load is used to indicate technical problems with the webpage that prevent users from viewing any LP content. 


Use the Did Not Load flag when: 


< Тһе МС ofthe landing page is a web server or web application error message and there is no other content on. 
the page: no navigation inks, no home link, no SC, and no Ads. See here for a Wikipedia page on different types 
of error messages. 

* The landing page is completely blank: no MC, no SC, and no Ads. 


‘Assign the Did Not Load flag based on the landing page, not the result block. 


Here is an example of a Did Not Load landing page. You cannot tell hat the landing page doesn't load by looking at the 
result block. 





‘Query and User Intent ‘Result Block and LP, Needs Met Rating, Flag Explanation 





Douglas Instruments Success 

in Protein 

rd This page displays a generic 404 
massage. There is no MC, SC, or 

[Ads on the page. 


Query: {douglas instruments) 


User Location: South Bend, 
Indiana 





User tant: navigate tota س‎ andine pid Nat Loma fas. 0 
homepage of his кебейе кетт. 


Fom Wa Fasian nase (Gt TE) 

















Неге are screenshots of other types of landing pages that should be assigned the Did Not Load flag: DNL, DNL2. 
You should not use the Did Not Load flag for: 
‘© Malware wamings, such as "Waming - visiting this site may harm your computer!" (example) 
‘© Pages that have removed or expired MC (е. expired classified isting, removed social media post, products ог 
«Pages hat are access because you ned a subscripton to view he MC 
15.5.2 Needs Mot Rating and the Did Not Load Flag 


All result blocks must be given a Needs Met rating. If the landing page truly doesn't load, assign the Did Not Load flag 
and rate the pago FallsM. True Did Not Load pages aro useless, 


‘Sometimes the page partially loads or has an error message. Give Needs Met ratings based on how helpful the result is 
for the query. Error messages can be customized by the webmaster and are part of a well-functioning website. 
‘Sometimes these pages аге helpful for the query. 


Note: If you are unable to evaluate the Page Quality rating of a Did Not Load result, you do not need to assign a Page 
Quality rating and can leave the slider at NIA. 
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‘Query and User Intent. 


Result Block and LP, Needs Met Rating, Page Quality 
Rating, Flags 


Explanation 





Query: [boys pink snow 
shoes] 


User Location: South 
Bond, Indiana 


User intent Find 
information about or 
purchase boys snow 
‘shows, 


Boys Snow Boots & Winter Socks | 
Nordstrom 

торошот som ки Boye 

sos end Sen be mee 





anc a) Fe ngage FW NILSEN 


Needs Met: The MC has an error 
message, but the LP has a lot of SC. 
However, the page has no information. 
about boys pink snow shoes and is. 
unhelpful foc the query. 


Page Quality: This LP is on a wel-known, 
reputable merchant website. Despite the 
error message, this page does a nice job 
of explaining the issue and providing а 
search box, including helpful search tips 
and customer service information. 





Do not assign the Did Not Load fag 





‘Query: [ible passages] 


User Location: South 
Bend, indiana 


User intent: Find 


specific passages in he 
Bile. 


Bible Gateway 
penne eatery cond 
Mabietend- Une you om digital амбу 
brary on Bibie Gateway Bowen Products Due 
Во; Day вве ноти news 








anc) Frei Language ы. анада ть: 


Needs Met: In spite of the customized "No 
results found” message on tha LP, it has 
inks t all passages in the Bible, organized 
by book. It would be very helpful for most 


Page Quality: This page is a wellknown, 
reputable reference website. In addition io 
iis to all passages in the Bible, this eror 
Page provides а search box as weli as a 
Suggestion explaining howto refine 
searches, 


Do not assign the Did Not Load flag. 





Query: [broadway 
seke] 


User Location: Nyack 
New York 


User intent: This isa 
Know query or Do 
query. The user wants 
о search prices andlor 
purchase tickets fo a 
Broadway show in New 
‘York Ciy. 


© Mpumateoursneut reet 


Click HERE for free broadway tickets! 


macleious ste broadway tickets phishing special 
otier -malware warming 


рок ha Fre nage Ы. андаа N 


Clicking on this result block leads to a 
malware waming. You can assume that 
the page is matcious without continuing to 
the page. As discussed in Sections 7.4.6 
‘and 13.6, you should assign a FallsM 
Needs Mat rang and Lowest Page 
Quality rating. 


Do not assign the Did Not Load flag. 








‘Query: {paranormal 
activity] 


User Location: Santa 
Rosa, Сайота 


User Intent: Learn more 
about the 2007 fim 
Paranormal Activity or 
the associated fim 





© hesipatasomabangoiz eom 


Paranormal Activity | Discussion Board. 


Welcome ta our Paranormal Aci discusion board 
Come discuss the fim, he series, or your vm personal 
‘encounters with paranormal 


iu uut 


For i| Fag Lag ма TR ar] 








This page cannot be reached because the 
site's server IP address was nol found. 


There is no MC, SC. or Ads on the page. 
This result merits а FailsM Needs Met 
тайла, and no Page Quality rating is 
required 


Assign the Did Not Load flag. 
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15.6 Additional Flags in Some Rating Tasks 
‘Some rating tasks may ask you to identify Upsetting-Offensive and/or Not-for-Everyone results. 


Mark content that may be upsetting or offensive from the perspective of a typical user in your locale as 
Upsetting-Offensive, keeping in mind that people of all ages, genders, races, religions, and political afllations use the 
Internet to understand the world and other points of views. 


Mark content that may be unpleasant or uncomfortable for some people in your locale (e.g. content that may not be 
appropriate in a public space, professional environment, or school) as Not-for-Everyone. 


16.0 Rating Queries with Multiple Interpretations and Intents 


‘Some queries really only have one meaning. Consider the query [iphone], English (US). There may be diferent user 
intents for this query (research iPhones, buy an iPhone, go to the iPhone page on Apple's website), but all users are 
basically refering to the same thing: the phone made by Apple, Inc. 


‘Some queries truly have different possible meanings. Consider the query [apple] English (US). Some users may want to 
find more information on the computer brand or the fruit. We refer to these different meanings as query interpretations. 


When giving Needs Met ratings for results involving different query interpretations, think about how likely the query 
interpretation is and how helpful the result is. 


‘© Avery helpful result for a dominant interpretation should be rated Highly Meets, because itis very helpful for 
many or most users. Some querias with a dominant interpretation have a FullyM result. 

* Avery helpful result for a common interpretation may be Highly Meets or Moderately Meets, depending on 
how likely the interpretation is. 

‘© A very helpful result for a very minor interpretation may be Slightly Meets or lower because few users may be 
interested in that interpretation. 

* There are some interpretations that are so unlikely that results should be rated FailsM. We call these "no. 
chance” interpretations. 


16.1 Rating Queries with Both Website and Visit-in-Person Intent 





‘Some queries have two possible strong intents: 
1. Go to the website intent: in order to, for example, find out information, buy something online, make a reservation, 
schedule an appointment, interact with customer support, or fulfil some other need that can be satisfied online 
2. Visit-n-person intent: user wants to visit the store, business, etc. in person. 


We know the user intent is to accomplish one or the other, but it is unclear which one the user wants. For these queries, 
result blocks that only satisfy one intent should NOT get a Fully Meets rating. 
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Query and User intent | Result Block and LP Needs Met Rating and Explanation 
E 
Б: 
= — 
а a ot 
‘Query: [target] E 
The result block shows three popular 
User Location: "T. Target locations in Jacksonville, with 
Jacksonville, Florida IRA information that is especially helpful 
for users who want to visit the stora. 
User intent: There are 57 This fulfils the user intent to find a 
wo possible strong user еә. nearby Target location. 
intents: most users 
probably want to visit a 
‘nearby Target location or 
до to the website to shop. 
‘online, research. 
products, find prices, etc. | Target т 
Saray regen шше тот The result is very satisfying and fulfills 
Б the user intent to shop onne or 
otherwise use the website. 
F Y 
‘Query: [amv] Í: "| г 
User Location: та The result block shows two nearby 
‘Belmont, California = FE DMV locations, with information that is 
et ‘especially helpful for users who want 
User Intent: There are o visit the locations. This fulfils the 
two possible strong user 5 mm ‘user intent to find a nearby DMV 


intents: most users 
probably want to visit a 
nearby DMV location or 
(до to the DNV website 
о renew a license, pay a 
fee, find some 
infomation, etc. 


location. 

















Tho result is very satisfying and fulfil: 
the user intent to find information or 
otherwise use the website (ag. lo 
review a icense, pay a fee) 
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‘Query and User Intent | Result Block and LP Needs Mot Rating and Explanation 








The result block shows three nearby 


‘Query: feibank] i 
Citibank locations in the user location 
User Location: Palo D oe of Palo Alto. The information is 
Ala California. ‘especialy helpful for users who want 
to visit the bank. This fulfils the user 
User intent: Thera are aes Intent to find a nearby Citibank. 
~ location. 


wa possible strong user = 
intents: most users 

probably want to visit a E 
hearty Clibank location. 

Jor go lo ihe websiteto | cru sign on 
bank onine. p жее 





















The result is very satisfying and fulfil: 
he user intent to do online banking or 
otherwise use the website. 














170 Specifi 





of Queries and Landing Pages 


‘Some queries are very general and some queries are specific. Here are some examples that compare levels of specificity 
of English (US) queries: 
































Query More Specific Query Even More Specific Query 
клап taining room chair] [kea “henrksdar' highback upholstered chair] 

ary harvard ibrary] harvard anthropology brary] 

Interview questions} interview questions for teachers] practice interview questions used for teach for america] 
restaurants] chinese restaurants] takeout chinese restaurants in downtown Austin) 
[cotes shops] fetarbueks] [red rock coffee mountain view] 





Results for specific queries are easier to rate on the Needs Met scale because we know more about what the user is 
looking for. Giving a Needs Met rating for results for general queries can be difficult. As always, your rating is based оп 
how helpful the result is for the query, not the specificity fit. 





When the query is a broad category, such as [cafes] [restaurants] [hotels] [books] [tourist attractions in paris] еіс. popular 
and prominent examples may be considered very helpful. Please do web research to help you understand what is 
popular and prominent in diferent locations. 
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‘Query and User Intent. 


Result Block & LP, Needs Mat & Page Quality Ratings. 


Explanation. 








‘Query: [credit cards] 


User Location: 
Phoenix, Arizona 


User Intent: Users are 
probably looking о sign 
up for a credit card 
Janine, or want to 
research credit cards 
before signing up. 


Credit cards & Rewards credit cards| 
Visa USA 

usa viaa con- cards/eredit-cards/index ja 
Мове fing - Whether you are looking fora. 
premium rewards card or a aciona! credit card, 
Ува has something to your needs 

Apply fora Credit Card = Apply 


Even though there аге many options, а 
page fr an individual company tnat offers 
‘ferent cards could be very helpful 


Needs Met: This LP is more specific than 
the query, but сом stil be helpful far 
many or most users because Visa is a 
popular credit card company. 


Page Quality: Visa is a credit card 
company with high E-A-T that affars 
services for credit cards, banking, ele. and 





sreap oom 

ass Ve шз ' vem | “rer [has a good reputation. High to Highest 
s an appropriate rating. 

Credit Cards and Credit Card Offers - en paus nan ка man) coke, А 


Apply Online | Discover Card 
ips wu discover comveredh-cards/ 

Мае полду - See al the ways you can eam and 
enjoy credit card rewards with the Discover it credit 
card the game changing cash 


page for an individual company that offers 
(йегет! cards could be very парі. 


Needs Met: This LP is more specific than 
the query, but could stil be helpful far 
many or most users because Discover is a 
popular credit card company. 


Page Quality: Discover is a company with 
high E-A-T that offers services far credit 

cards, banking, ete. with a good reputation. 
High to Highest is an appropriate rating. 





Best Credit Cards of 2015: Reviews, 
Rewards and Offers - NerdWallet 

werm тиде corhe-bec credit cards 
Mobile. french - NerdWalere eredi card expert 
rank the best eret cards out there. fers include 
noe cash back ards vith up ш 5v. 





П ЕЕЕ 





Tris page offers a list of ор credit cards in 
a variety of categories. 


Needs Met: This LP fits the query. It 
would be helpful for most users. 


Page Quality: This website heips 
consumers make money decisions. It has 
а good reputation, high E-A-T, and has 
been recommended by several major 
newspapers. High is an appropriate 
rang. 








Union Plus Credit Card for Union 
‘Members 

Wwwunlonglus org. credit/eredit-card 

Море елду - The Union Plus Credit Card ie 
designed for union members. The credit card offers 
financial hardship assistance, trike 


ктт» حم‎ 





Tis page describes а crdi card that 
requires union membership. 


Needs Met: Since the credit card requires 
‘union membership, the page would be 
helpful far some users. 


Page Quality: This company is an expert 
on its own credit card, which is issued by 
HSBC Bank, а financial services 
‘organization wih a good reputation and 
gh E-A-T. High is an appropriate rating. 
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Query and User Intent. 


Result Block & LP, Needs Mat & Page Quality Ratings. 


Explanation. 








Query: hotel] 


User Location: 
Jacksonvile, Florida 


User Intent: Users are 
probably planning a trip, 
ut this query is very 
general and vague. 


Hotels, Hotel Rooms, Discount Hotel 
Rates & Cheap Motels on Orbitz 

кин ойт солуна 

obie rend - Find discount ota motels and 
heap otl rooma with hotel reviews on bt Get 
эх юнен oti rates on our. 





This в a popular travel aggregator website, 
and the hola page on the sta can help 

users find hoteis in the U.S. Users can 
rad reviews, compare hotels, or make а 
reservation 


Needs Met: This LP fits the query. It 
would be very helpful for most users. 


Page Quality: Orbitz is a popular website 
wih high E-A-T and a good reputation 





3 ا‎ ghe an appropriata ае, 
тнв в avery popular hoe chal wth 
А notas avaiable in tne тарту ol the US. 
Marriott ai mary aaran poo pares, 
wern marriott con default mi 


Mobile-fiendly - Browse and book hotel rooms 
that suit your needs and style and start your next 
adventure with Marriott, 


n бз te 





Even though the list of possible hotel 
chains в long, the homepage of an 
individual chain that offers different prices, 
features, and location options could be 
vary helpful 


Needs Met: This LP is more specific than 
the query, but t would stil be перы far 
many or most users because the Marriott 
isa popular chain of hotels. 


Page Quality: The Mariott website gives 
information on Mariott hotels а popular 
chain of hotels. Marriott has а good 
reputation and is an expert on Mariott 
hotels, making the information on this page 
highly authoritative. High to Highest is 
an appropriate rating. 











Emeryville Hotels near Berkeley CA | 
Courtyard Hotel in ... - Marriott 

warn marriott comv hotel]. /osenv enu 
Мове непау - 4 days soo This Emeryville, CA 
hotel near Berkeley offers a moder lobby with 
exible work spaces re Wi-Fi 

Hotel Details - S565 Shelmcund Stet 








Tis в the webpage of the Mariott 
Courtyard hotel in Emeryile, California. 


Needs Met: The LP в too specific for the 
query, but this В a wellknown brand and 
users can navigate to other Marriott hotels 
from this pago. Few users would find this 
page helptl 


Page Quality: The Mariott website gives 
‘information on Mariott hotels а popular 
chain of hotels. Marriott has a good 
reputation and is an expert on Marriott 
hotels, making the information on this page 
highly authoritative. High to Highest is 
[an appropriate rating. 
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Query and User intent 


Result Block & LP, Needs Mat & Page Quality Ratings. 


Explanation. 





Query: target] 
User Location: 
Jacksonville Florida 


User Intent: Go to 
target.com or fnd a 
nearby Target store. 


m um 
E ж 
эм te 





а). Мо Page Quality rating 
E E 


Needs Met: The result block shows three 
papular Target locations in Jacksonvile, 
with information that is especially helpful 
for users who want to visit the store. 





Target 


‘The LP is the Target website. 


Needs Met: The result is very satistying 
for mobile users that want to go to the 
website. 


Page Quality: Target is a popular 
shopping website in the U.S. with high 
E-AT and в trusted by users for online 
purchases. High to Hight is an 





Find Store - Target 
metere cator rd tren 

Noi игу fata eoe Target fr atr 
earch ors tore cr мун al etre cttm 
ny show ишек wih Alo 


тт, GU 





Needs Met: The LP is more specifie than 
tha query, but many or most users would 
be interested in this page, 


Page Quality: Target is the expert about 
where its sores are located. This page is 
tha most authoritative page on this topic 
High to Highest is an appropriate rating. 











Electronics : Target 
miargetcomvcielectonicsi- N-Srg 
oiler - Buy electronics onine from the 
top brands at Target rom TVs and cameras to 
computere and tablets. Free shipping on. 

TVs home theater -Computers & Office 


سے وی 








The LP is the "electronics page on the 
Target website. 


Needs Met: The LP is more specific than 
tha query, but many or most users would 
be interested in this page. 


Page Quality: Target is a popular 
shopping website in the U.S. with high 
E-A-T and is trusted by users for online 
purchases. High to Highe is an 
appropriate rating. 
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‘Query and User Intent 


Result Block & LP, Needs Mat & Page Quality Ratings. 


Explanation. 





(Query: [target] 


User Location: 
Jacksonvile, Florida 


User Intent: Goto 
target.com ar find a 
nearby Target store. 





Target 
Corporation 








|. No Page Quality rating 
U^ required 





Needs Met: While this result block has 
soma helpful content, most people 
searching for Target are not looking for 
the stock price or the names of the CEO 
and Founder. Slightly Me 

{Slightly Meets* is an appropriate 
rating. 











Women's Jegging Black - Cherokee 
Target 

‘target сопу, women jeggig- Macc. 
Mobile- enhy -Jan 42015 Find product 
information, ratings and reviews for a Women's 
egin Black - Cherokee. 


e‏ و 


The LP on the Target website is for a 
specific type of woman's pants 


Needs Met: The LP is much more specifie 
than the query and would bo helpful for 
vary few users. 


Page Quality Target а popular 
shopping website in tha U.S. with high 
EAT andis tisted by users for online 
purchases. High to High? is an 








appropriate rating. 
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‘Query and User Intent. 


Result Block & LP, Needs Mat & Page Quality Ratings 


Explanation. 








‘Query: [chicken recipes] 


User Location: Austin, 
Texas 


User Intent: Users 


probably want to make а 
eken dish and ar 
looking fr some recipes 
чо choose from. Users 
probably expect and 
anta list of recipes. 





Chicken Recipes: Food Network 
Recipes and Cooking: Food 

ow оолак con dicken huri 
Results - 10 of 800 - Find chicken recipes, 
videos and ideas tom Food Network 





This LP has many chicken recipes (with 
views) on a popular recipe website 





Needs Met This LP fts the query. 1t 
wouid be very helpful for most users. 


Page Quality: Food Network is a popular 
website with high E-A-T and a good 
reputation far providing high quality recipes 
and cooking information. High to 
Highest is an appropriate rating. 





Chicken Parmesan Recipe : Tyler 
Florence: Food Network 


Ма «=ч feodrevarccom лу. 


г: eg 
oe Fray он ar Ferner 


esc Chekan Parmesan recipe fon 


‘This LP has a single recipe for chicken 
parmesan. 


Needs Mot: This а popular type of 
chicken recipe on a popular website, but 
the LP в mare specific than the query. 
Some or few users would find this page 
терм. 





Page Quality: Food Network is a popular 
website with high E-A-T and a good 
reputation for providing high quality recipes 
and cooking information. High to 
Highest is an appropriate rating. 





Fried Chicken Recipes - Allrecipes.com 
maliedpesconyeipel./ccken ted 
Маме Fendi - See more han BO delicious 
authenti Мей chicken recipes with photos and 
dps... Stal Picks Honey Fried chicken 





This LP has over 25 recipes for ried 
chicken, a popular chicken dish. 


Needs Met: Evan though there are over 25 
diffarant recipes on the LP, they are all for 
the same basic dish. Therefore, this LP is 
also more specific than the query. Soma 
Jar few users would find this page helpful 


Page Quality: Alecipas com is a popular 
website with high E-A-T and a good 
reputation for providing high quality recipes 
with lots af reviews. High? to Highest is 
[an appropriate rating. 











Chickenrecipes 
этн chickerrecipes. conv 

hickenrespes com has been informing vistors 
about topics such аз Chicken Breast Recipes, Easy 
Chicken Dishes. 











Tis в а fake search page for chicken 
recipes. 


Needs Met: t should be rated FailsM as t 
Would be heiph for no users. 


Page Quality: Ads should never disguise 
hemeelvas as the MC ofthe page. Pages 
wih Ads that are designed to look ike MC 
do not have a beneficial purpose. Since 
this page deceives users, it shouid receive 
the Lowest rating. 
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‘Query and User intent 


Result Block & LP, Needs Mat & Page Quality Ratings. 


Explanation. 








Query: fsmithsonian air 
and space museum 
store] 


User Location: 
Washington, DC 


User Intent: Go to he 
store website or find 
information, such as 
opening hours, about tha 
museums stora in 
Washington, DC. 


Mussum Store | National Air and 
Spaco Museum 

Shscnan поп авла, 

‘ina tne pere у к someone репа Me 
Museum see oe 





‘This LP is tha official website for the 
museum store. 


Needs Met: The result is very satisfying for 
mobila users that want to до lo tne 
website. 


Page Quality: This is the website ofa 
popular museum stora with a good 
oputaton and high E-A-T. The museum is 
the expert on information about the store, 
including hours, phone numbers, 
membership information, etc. Might to 
Highest is an appropriate rating. 

















No Page Quality rating 
U^ required 





Needs Met: While the result is about the 


museum, the hours, address, and 
‘rections are the same as the museum 


store. This information is helpful for users 
searching for the store. 








Volunteer | National Air and 
Space Museum 
ung jou е aes 














Tha LP has information about how to 
volunteer at tne museum. 


Needs Met: The LP is more specifie than 
he query and would be helpful for few 


Page Quality: This is the volunteer page 
fera popular museum with a good 
reputation and high E-A-T. The museum is 
the expert on their own volunteer 
opportunities. High to Highest is an 
appropriate rating. 
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‘Query and User Intent. 


Result Block & LP, Needs Met & Page Quality Rating 








Query: fsmithsonian air 
and space museum 
store 


User Location: 
Washington, ОС 


User intent: Goto the 
store website or find 
information, such as 


opening hours, about tho 
museums stora in 


Washington, DC. 

















No Page Quality rating 
U^ required 





Needs Met: This is not the museum. 
requested in tha query so the result Fails 
to Meet the user intent 
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18.0 Needs Met Rating and Freshness 


‘Some queries demand very recent or “fresh” information. Users may be looking for "breaking news,” such as an important 
event or natural disaster happening right now. Here are different types of queries demanding currentirecent results. 





“Type of Query Example Queries Explanation 


Assume users need the information right away. 





Breaking news" queries tomado] [tsunami] Imagine someone who needs immediate weather information because 
a big storm s coming. Information about last year's weather would 
not be het. 





Assume users are looking for the most recent ar current information 
about the event. 

Recurring event queries, 

such as actions, sports 
events, TV shows, 
conferences, oc 


olympics, [american о], 
[redsox schedule, ax forms} 
elections) 


For example, ifthe Olympics are happening right now, users 
searching [olympics] want information about the current Olympics, not 
results from years ago. ithe next Olympics are a few months away, 
users are probably interested in the 

upcoming Olympics. 





[population of paris, {amount of 
us. dett [airfare from ny to stol, 
nex! federal holiday] 


Current information 
quotes 


Assume users are looking forthe most current information, such as 
prices or airfare. 





iphone), Бауска camry], [windows | Assume users are lacking for information about the most recent 


Product queries سم‎ seem] modellversion for hese product queries. 

















When a query demands recent content, only pages with current, recent, or updated content should get high Needs Met 
ratings. For these queries, pages about past events, old product models and prices, outdated information, etc. are not 
helpful. They should be considered “stale” and given low Needs Met ratings. In some cases, stale results are useless 
and should be rated FallsM. 


For some queries, there may be "newsy" or recent information user intent, as well as more “timeless” information user 
Intent. Users issuing queries for celebrities or politicians may be interested in biographical information, or users may be 
looking for the latest news or gossip. 


Freshness is generally less of a concern for Page Quality rating. “Stale” pages can have high Page Quality ratings. For 
‘example, some highly reputable news websites maintain "archival content. However, unmaintained/abandoned "old" 
websites or unmaintained and inaccurate/misleading content is a reason for a low Page Quality rating. 


Note: The date the page was created may be diferent from when the content was last updated or modified. When 
content is updated, the page will sometimes show the date of the update, not the date the page was created. Some 
websites always show the current data, regardless of when tha content was last updated. If you are curious about the 
content of a page, see here to try the "Wayback Machine" on the Internat Archive. Not all pagas are available, but this 
tool may help you understand how some pages were created and how their content has changed over timo. 
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‘Query and User Intent 


Result Block and LP, Needs Met Rating, Page Quality Rating 


Explanation 





Query: [nasa mars 
landing} 


User Location: San 
Francisco. Самота. 


User Intent: On the date 
this example was 
‘writen, NASA had just 
landed the “Curiosity” 
rover on Mars. Most 
users are probably 
looking fr breaking 
news and pictures about 
the event. 


Curiosity Rover Lands Safely on Mars- 
NYTimes сот 

monda nine cone ow 

‘ped eon teen teer 











‘This LP has an article about the 
“Curiosity” rover landing, 
published on the sama day as the 
landing, which is he day this 
‘example was writen. 





[NASA's Curiosity Rover Sets Of for 
Mare iion “WV Tinea 
eere on ai te oe. 





This LP has an article about the 
launching ofthe “Curiosity rover, 
published rina months before ils 
landing on Mars. Tha content an 
this page would be considered 
sale for the query 





Query: los angeles 
тата 


User Location: Los 
Angeles, California 


User Intent: Users are 
locking for current 
information about afe 
‘colton in Los 


Angeles. Evan pages 
about traf conditions 


the day before would be 
considered stale forthe 


query. 


Your commute: Live LA. rff 
conditions ILA NOW Los Angeles 
etii cel. И -ar 
уена 
“Tener deno 











This LP shows trafic conditions in 
Los Angelos on August 7, 2012, 


which is the day this example was 
‘ween, 




















‘Query: [nobel peace 
prize winner] 


User Location: San 
Diego, California 


User intent Find the 
пате ofthe most recent 
‘winner of this prize. This 
is a Know Simple query. 
Айке time this example 
жаз originally written, 
2011 was the most 
тесеп year that he 
Nobel Peace Prize was 
awarded. 





c Ca uu T 
= a ee et 
Your commute: Live LA rae 
conditions| LA. МОМ Los Angeles 
“Seti The LP shows trac conditions in 
ыыы етае агаа. Los Angeles оп August б, 2012. 
Шр» T do uv lets fr be 
very even though tis ely one 
day o 
—A— еру 
T DUM 
Nobel Pesce Prize Awarded to Three 
Activist Women 


‘ery com 201 etl pne 
тише bal Peace Pro a dto me ji 
Sora citer 2017) 





This LP on the New York Times 
website has information about the 


most recent winner of this prize, 
based on the date when this 


example was written. 








BBC NEWS Europe | Nobel for anti- 

poverty pioneers 

ceria эи Mura Vaan тилден, 
e Grameen boru tu tren m 














‘This LP on the BBC website has 
information about he 2006 winner. 
ofthis prize. This page is stale 
for he query. 
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Query and User Intent [Result Block and LP, Needs Mot Rating, Page Quality Rating | Explanation 
Arthritis: MedlinePlus 
‘mmedineun gwar nt 
Query: artis) you feni pained fes ауди dy or have 
‘rouble moving around, you might have arthritis Medline Plus is an authoritative 
User Location: еее ebste al regulary updates йв 
Phiadelpha orient. 
Pennsylvania 
— IE 
User intent Users are |= Me ue Wem ме 
ooking frormalan 
shout thie боена. Arthritis Drug Approved for Expanded 
осле Сене Use - New York Times. 
ongoing research done | sore лрите, эпилог лла 
n релна ard awae ay 20 nF pred ie rie lle wed Ba: 
Н КЫЫ? | unc mon boson ug hentai considered stale tor is medical 
шона, up-to-date qm, 
tration оп aris 
ا و‎ 
Franklin D. Roosevelt - Biography - US. 
President - Biography сот, 
Query: Ihen Waê | gap con ann roose 
franklin d roosevelt born] | вот on January 30, 1882, in Hyde Park, New York, 
Fal D emer wes cen mih lom 
фм: 11921, He became the 32nd U.S, president in 1933, EM aman Lade 
Chicago, Ilinois and was the only president to be elected four times. жюк. 
User intent: Users want 
to nd he wel 
understood fac кте» CDU 
We couldn't fnd a stale page for this quan NA 

















19.0 Misspelled and Mistyped Queries and Results. 
194 Misspelled and Mistyped Queries 


You will notice that some queries are misspelled or mistyped. Here are some examples of queries that are obviously 
misspelled: 





Misspelled Query Query Interpretation. 


federal expres), Engish (US) The only reasonable query interpretation is the company named Federal Express 








[new england patris] English US) | Тһе only reasonable interpretation is the NFL football team. 





Ivonce knowles], Engish (US) The опу reasonable interpretation is the famous singer/actress, Beyonce Knowles. 











‘Some misspelled or mistyped queries are more dificult to interpret. Use your judgment and do query research. 


For obviously misspelled or mistyped queries, you should base your rating on user intent, not necessarily on exactly how 
the query has been spelled or typed by the user. 


For queries that are not obviously misspelled or mistyped, you should respect the query as written, and assume users 
are looking for results for the query as it is spelled. 


Copyright 2021 150 





192 Мате Queries 





Consider the query john stuart], English (US). There is a very famous Jon Stewart, the comedian and former host of a 
popular U.S. television show. However, we should not assume that the query [hn stuart] has been misspelled. There 
аге many people named John Stuart. We will respect the query as written and assume the user is looking for someone 
named "John Stuart." 





Query and Possible 








Кеш Result Block and LP, Needs Met Rating, Page Quality Rating Explanation 
The page fists ай 
Micheal Jordan profiles | Linkedin күсе 
аре sjana inked соту. ев) or ordan" Linkedin ts a 
Мове тау View the profes of pofesiorls very papular social 
amed Micheal Jorden on Linkedin There ме 25 networking website for 
prolesslonls named Мелем professionals in 1e U.S. 
Query: fits огап. ‘This result would be very 
Engien (US) help for many or mast 
users tying to fnd 

















User Location: San سب‎ Formaten about 
Francisco, Сайта | gig з ко" Bona ge MM enhn 
Picea oria." 
[user intant: There а 
many peopl named 
Mein Micheal Jordan | Linkedin 
нинин facio. | own oin rd 
Ни ено forte Msn nia. Pete Seber rion нөр 
Micheal Jordan should Оса Supporter at Fert Stat UM. Church; Micheal Jordan, a 
Schaal кадап shuld iat Fr eet ted акам 
тапш тышы | манон описа Supver [eer arene 
ass prominent pare Georgia 
famed Micheal Jordan 
و‎ ee 
coe id in| TT e Um UE 
лае 
atveteiceletriy we wa. | NBA.com: Michael Jordan Career Stats 
een Jordan | www nba.com/history/.../ordan_stats htm. 
pe Ful Name: Michael Jer] Jordan Born: 2/17/63 in 
roo. NY. High Schoot: Laney “Ніно, it Jens aise 
NC) College: North Prof page on ne NBA 


basketbal website. 





Aa. UM 

















20.0 Non-Fully Meets Results for URL Queries 


Raters sometimes ask the question, "For a well-formed working URL query, are the only acceptable Needs Met ratings for 
a result either Fully Meets or Fails to Meet?” The answer is по. There can be other helpful results for URL queries, 


‘Some users issue URL queries to find information about a website, such as reviews or recent news. We recommended 
this to you as one method of reputation research in the РО guidelines. Real users do this too. Results that give reviews 
and reputation information can be very helpful for a URL query. 


However, websites that offer usage statistics about a website are not usually helpful results for URL queries. Most users 
arent interested in this kind of information. 
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(Query and User Intent [Result Glock and LP, Needs Mot Rating, Page Quality Rating | Explanation 
Peter Bam 
тшм ы a чини fh ont hs LP а he target of he query. 
ACACIA] "77% 
Query: 
[poterybar com Upra 75% ot Ponary Ban Coupons, Prana Onine coupon cades are very 
бишне Wee b aada popular һе US. and hs LP 
(unir Loads Pal (топ a wel known coupon e) 
ы e vert retra compe a аиан e id 
rore coner s ay 23 Киш тон: Sve Up purchases on the Pottery Bam. 
Userintent:Potiey | соттон ™ Weste. Users may be intrested 
Bamisahome în coupon codes апа this would 
ishing tore, Users be very help for those shopping. 
may want to goto nine at palteybam com, 
аган سيل ي‎ 
ebane, do шшк. or | Ele de эы [especialy едет shoppers 
epey Foy Ban Ress Consumer Reviews ot 
Panaypa com | Sober “his LP (kom a wellknown 
onsen тен теи теген ste) has reviews of he 
AREA pottenbam com website. Users 
Dene auf ioe miei E 
evens and Campie reviews when considering 
‘wheter to make purchases at 
oleam com, especialy new 
eg [storms 
Rate My Professors - Review Teachers 
and Professora, School 
mme 
Calpe pene dri cuc bd his LP is the target of the quer 
nan tac бета а 
Se cá 
lisi ттт G0 
Memyprfessorscom] | RateMyProfessors com Rankings 
User Location: Los 1 "088559 е Rence Sd The LP is a Hufington Post 
Angeles California  |мыккешу їл 217 Te riino ater article dated August 31, 2012 
уок to boe pemphigus а [about ratemyprofessors com. 
ger intent: бою | Amena woe Uiar opaan. The ate was newly published 
Б экеп was added о ha 
Engr PT idees. Some users might be 
fuente can rate the | nef tma [interested inthis article 
e T T. xr 
Токто nere quis ready 
Бетсен The LP в an article dated Apri 
Er 17, 2006 about the 
иил omyprolessore com website. 
eer Few of no users would бе 
interested in ts outdated 
يي‎ e malen. 
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210 Product Queries: Importance of Browsing and Researching 


‘Some product queries, such as [ipad reviews], have a clear information-seeking (Know) intent. Other product queries, 
‘such as [buy ipad}, have a clear purchase (Do) intent. And some product queries, such as [ipad store.apple.com], have а 
clear navigation (Website) intent. However, most product queries don't obviously specify one type of intent. 


Keep in mind that many users enjoy browsing and visually exploring products online, similar to window shopping in real 
life. Give high Needs Met ratings о results that allow users to research, browse, and decide what to purchase. 


Users may not always plan to buy products online that they are browsing and researching, for example, cars or major 
appliances. Even though the ultimate goal may be to purchase a product, many other activites may take place first: 
researching the product (reviews, technical specifications), understanding the options that are available (brands, models, 
pricing), viewing and considering various options (browsing), ete. 


Important: Page Quality ratings for product results need extra care and attention. 


Often, the results for product queries are YMYL pages. Users need high quality information from authoritative sources 
when researching products, especially when products are expensive or represent a major investment/important life event 
(6.9. cars, washing machines, computers, wedding gifts, baby products, hurricane shutters, large finess equipment) 
When buying products, users need websites they can trust: good reputation, extensive customer service support, etc. 
Results for product queries may be important for both your money and your life (YMYL)! 





























uU ee ГҮҮ 
ттин 
PCMag.com This LP has comprehensive 
| User Location: Los жүн рота corvreviews/HOTY information about diferent types 
[егег crono odit m ree к аре 
чынлы, LCD, LED), as well as reviews. 
etc T quay Ae nto Tr 
[кесе ере. esta Leer 
E EXTA perisse н ИЩ 
Er | ааа [шшс 
ټی د‎ E 
КЕ 
ИНЕ EE A 
egg terre | oteaan Sen 
peter lo MM ais hop ap for 
шшк | ет о А o E atero or inla 
e аа БА е АЕ, 
rere cambi iie ele m ate TT This 
ert Ta cm оче рн 
beers EE. 
устан 
и teal e 








Query: caf] Cuff Links for Men & More Men's 
Vasrtoeaton:ios | аон This LP has a largo cation of 
Angeles, Сайыы | Чори enay герт ia & afin rom a very reputatie 


‘taller. There are many optons 
and styles. This is a good page 
for browsing. There are helpful 
features atthe top to choose 
styles. price ranges, c. 


‘mers jewelry at ойтеп com Stop of inks 
User Intent: Most users | te cips deus & necklaces Totaly 
probably want to find 

‘ferent cuffini styles 
and options to browse or 
purchase. "YEN MET E 
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Query and User Intent. 


Result Block and LP, Needs Met Rating 


Explanation 





Query: wist of fale ring pandora} 





User Location: Philadelphia, 
Pennsylvania 


User Intent: Tha user is looking 
for a specific ring soid by the 
Pandora brand and Ikely wants to 

"search the ring, see what it 
looks ike, find options to browse 
or purchase, etc: 




















This result has a variety ої 
images of the ring, showing 
ferent angles and materiais. 
Knowing what the ring looks ike 
is vary helpful to users who are 
researching the product and 
‘browsing options that ar 
avaliable 





22.0 Rating Visit-in-Person Intent Queries 


When there is a user location for a visitin-person intent query and a location has not been specified in the query itself, 
Such as [chinese restaurants] with a user location of Boston, MA, results in or near the user location are the most helpful 


How close is near"? The type of business and/or entity should be taken into consideration when deciding if the distance 
ofthe visitin-person result is too far. For example, most people are not wiling to travel very far for a gas station, coffee 
shop, supermarket, etc. Those are types of businesses that most users expect to find nearby. 


However, users might be wiling to travel а Ие farther for certain kinds of visit-in-person results: doctors! offices, libraries, 
specific types of restaurants, public facitis ike swimming pools, hiking trails in open spaces, etc. Sometimes users 


may accept results that are even farther away, such as a very specialized medical cinic. 


In other words, when we say users are looking for results “nearby,” the word "nearby" can mean different distances for 
different queries. As always, please use your judgment 
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22.1 Examples Where User Location Does (and Does Not) Matter 


As previously mentioned, the user location may not always change our understanding of the query and user intent. Below 
is one example where user location plays an important role, and two examples where user location does not matter. 





Query and User Intent. | Result Block and LP, Needs Met Rating Explanation. 





c d ‘This SCRE displays the complete and 
E N correct answer prominent in the result 


block, in a way that is easy for users on 
mobila devices to read. 








gE per rou van 


Note: Assume that the result shows 
current information for users at the ime 
bis example was writen. If you were 
rating this task, you would need о verily 
that the minimum wage rale in this SCRE 


is accurate lor tha state of Kansas, 
Query: minimum wage 


ate] 








us 





r Location: Wichita, | ООА 





к= p This is the official Kansas state website 
User Intent: Find out page on minimum wage for Kansas 


hat curent 
imum wage в. LE 
Deen sates the 

US have fret 





minimam wage rates. | mun wane taws in he states 
[Users might be eae aa А8 тнв LP has the minimum vage rales tor 
interested in the Sn mit: meer fe тин. all states. It is on the official U.S. 


minimum wage in their rita Department of Labor website, which has 

slate and how t high E-A-T on the topic, but it takes some 

compares to ather work to find the information specific to 
Kansas. 

places. 








Malu wage - Calloria Department 
fines Relations 
Ra ros aan egi i уйа вен Tha is the official Calloria state website 


= КЕТЕТ 


he 
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‘Query and User Intent. 


Result Block and LP, Needs Met Rating 


Explanation 





Query: [pictures of 
tens] 


User Location: 
Pittsburgh, Pennsylvania 


User Intent: Users are 
looking for pictures of 
kittens. This isa 
non-visitin-person intent 
query. Thera is no 
vicus user intent to 
ind pictures of kittens in 
Pittsburgh. The user 
location plays no role in 
ке Needs Met rating. 








The LP is a page ful of kitien pictures. 
The result block has а nice assortment of 
pictures as well that wouid highly забу 





pittsburgh pets - craigslist - Pittsburgh 
= Craigslist 
isbah craigs organe 

Mobi trendy рыза peta rigat. 

pets searches on as image: posted today. 
Tte ser аитат, 





This is a local isting of pets needing 
homes in the Pitsburgh area. There are 
no pictures of any pets cirectiy on this 
page and few pictures on the individual 
listings. 





Query: {googie jobs) 


User Location: Miami, 
Рона. 


User Intent: Users are 
locking fr job 
opportunities at Google. 








Googleplex 








кь 


The user в looking for a job at Google and 
thera is по information in this result block 
that wil help tha user explora job 
opportunities t Fails to Meet the user 
need. 





Google Careers 
pe eww google comsbowcarees! 

тые TEEN Адом Googe: Careers. Google 
Jobs Toke are on he Googe зебрі 
тон Stop by our олег around te ne 
Sear je Locatons-Teans and Rates 








‘This is Google's oficial page on careers 
and jobs at Google, whera users can 
search for curently open opportunites. 
This page is exactly what the user is 
looking foe Fully Meets the user need. 
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23.0 Rating English Language Results in Non-English Locales 


The following rating guidance is for raters in non-English locales. You may stop reading this section if your task 
language is English, such as English (US), English (IN), English (NG), etc. 





Your Needs Met ratings should reflect how helpful the result is for users in your locale. When the query is in the language 
of your locale, assume that users want results in that language. We know that you can read English (you are reading this 
document, but you should only give high Needs Met ratings to English results if users in your locale would expect or 
want them for a particular query. Unless requested by the query, English results should be considered useless if most. 
users in the locale can't read them. 


However, rating can be more dificult when the query includes English names, words, ete., or when it's unclear whether 
English results would be satisfying for a particular query. Please use your judgment and knowledge of your locale to 
determine the appropriate rating. 





Important: Please keep in mind that every locale will have unique considerations regarding the number and variety of 
languages (such as official languages, regional languages, local dialects, etc.) writing systems, and keyboard input 
languages commonly in use. While this guideline may not include examples for your locale, it is important that you 
represent users in your task location and culture in order to interpret the query and rate results. When in doubt, please 
assume that users would prefer results in the task language unless the query clearly indicates otherwise. 
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23.1 Examples of English (and Non-English) Results in Non-English Locales 


The section includes some examples using Hindi (IN) and Korean (KR) as the locales. In both cases, we cannot assume 


that users in these locales—i.e., Hindi-speaking users in India, or Korean-speaking users in Korea—are 





le to read 


Engish. Unless most users in the locale would be satisfied by English results for the query, we will consider them. 
unhelpful or even useless (FallsM) 











User Location: Delhi india 


User Intent: The user wants to 
d information 
about irat fe ef, an Indian artist. 








a DENTES Emm 
ram afa aat - afr aT 
eer AO 
RUE, icing cy zs 
-— EE снн ae helpful for Hindi (IN) users. 
imê 





Raja Ravi Varma - Wikipedia. the free 
encyclopedia. 
hipster m анада arg жы Rata 

тух тт Raja Ravi Varma Cat Thampuran 
(layla: пош cocido 29 дой в 2 
cer 1308) was an nian 
Leoni Vis Palace, Vadodara - Dan ayant 


So 





‘This Engish Wikipedia page 
about ira VR am has similar 
content to the Hindî Wikipedia 
page. Although there are some 
helpful images on the page, few 
Hind! users would be able to read 
his page. 





Query: pet SH] 
Locale: Korean (KR) 


User Location: Seoul, Korea 





User Intent: sel о is Barack 
‘Obama in Korean, so the user 
wants o find information about 
Barack Obama. 





et она Fla, 9а aee) aiat iet 








пн аа 





‘This Korean language Wikipedia 
page on Barack Obama is very 
lot for Korean (KR) uses 





‘en wikipedia. orgwk/Barack Obama. 
Мое any Barack Hussein Obama la 
an American ракав who served an he th 
President of ho United State rm 200910. 
2017, не ie ho frst Асап Amarican fo 


poo 














This Engish Wikipedia page 
about Barack Obama has similar 
antent to the Korean Wapedia 

page. Although hera aro some 

helpful images on the page, very 
few Korean users would be able 
to read this page. 
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Here are two examples where the query includes proper nouns typed in Latin script, such as famous people, places, tiles 
of books or fims, etc. For these queries, users would prefer to see results in the language of their locale. 





‘Query and User Intent 


Result Block and LP, Needs Met Rating 


Explanation. 





Query: [ronaldo] 
Locale: Hindi (м) 

User Location: Delhi, Inda 
User Intent: Tha user wants to 


nd information about Cristiano 
Ronaldo, a famous footballer. 


гет: dar - afar 
ps ope org wi = 


sarge gee, бейти deret йн. 
яа уйча жети, (1986-0205) 9 sat 
E 








Although the query was typed in 
Latin script, most Hindi users 
wouid wani to see information 
about this person in Hindi. This 
Hindi language Wikipedia page 
on Cristiano Ronaldo в very 
helpful far Hindî IN) users. 





Cristiano Ronaldo - Wikipedia, the free 
encyclopedia 


Mies en wkipeda org тї, Cision, 


ig FI Crtane Ronaldo асе santos 
Arero, GOIH [bom 5 February 1885) known as 
стапа Ronald (Portuguese 

‘Gemma Anson Messi Ronakto ary 


ыг 


This Engish Wikipedia page 
about Cristiano Ronaldo has 
‘Similar content ta the Hindi 
Wikipedia page. Although there 
are some helpful images on the 
age, few Hindi users would be 
able to read this page. 





Query: titanic 1997] 


Locale: Korean (KR) 


User Location: Seoul, Korea. 


User Intent: The user wants to 
purchase a DVD or fnd 
information about the movie 
“Tanic,” released in 1997 





Е 08У - Daum 4% 

mmovie daum net/m/detal/main deum? 

ces ла terze жердо PIE HS ao 
M Si аз dera аи чеч ваце 
sis ещ essa 





‘Although the query was typed in 
Latin scit, most Korean users 
would expect to see Korean. 
language reviews or Korean 
stores for purchasing the movie, 
‘This Korean result on a Korean 
website is very helpful 





Titanic - IMDb 
mimdb.com/ie/non20338/ 

жже Rating: 77/10 - 640553 votes 
Mobsle-triendiy - Tani -- This made-for-TV 
movie is the tale of the famous Tani ~- Jack 
discusses his view of the world with he frat 








This Engish language landing 
page has helpful content for users 
who can read Engish. Although 
ега are some helpful images on 
the page, very few Koraan users 
wouid be abie to read this page. 
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Here are two additional examples where the query was typed in Latin script (including some English words), yet users 
would stil prefer to see results in the language of their locale. 





‘Query and User Intent. 


Result Block and LP, Needs Met Rating 


Explanation 





Query: [mahia cricket sri lanka 
ka match] 


Locale: Hindi (IN) 


User Location: Delhi, india 


User Intent: The user wants to 
fd information about a women's 
Cricket match against Sri Lanka 
on February 17, 2016. 


лб йити жинби te 
Es 


m" 





Although the query includes some 
English words, itis a Hindi query 
typed in Latin Script. Most Hindi 
users would expect to see Hindi 
results, ike this helpful acl, 








Indian Women's Team Defeats Si Lanka to 
Seal OD Series- Cricket News- Sports 
sport ndtv com cie 2592 nd. 


‘cet eam defeated Lanka by sie мсн о 
betes in 


dá 








This Engish language landing 
page has helpful content for users 
who can read English, but few 
Hindi (IN) users would be able to 
read this page. 





Query: [samsung taber 
Locale: Korean (KR) 

User Location: Seoul, Korea 
User Intent: The user wants to 
purchase, find information, or go 


fo the Galaxy tablet page on the 
Samsung website. 





Ard BRIA 

www samsung сопу. /table./SHW.MB. 
эни 2i - SAMSUNG RINE SKTSHW- 
M3BOSFKASC, Samsung GALAXY Tab 101 
rope 

29 -Sgu cepa опа 





Although the query was typed in 
Latin script and includes some 
English words, most Korean 
users would expect to see the 
Korean page on the Samsung 
website. This Korean result is 
vary helpful 








Galaxy Tab: Android Tablet - Portable & 
Light Tablet! Samsung 

ww samsung coniun gala tab 
Webley Met he Samsung Galasy Tab 
mily insluding Galaxy Tab 10 1.8 9 and 7 O Plus, 
Sart by size or camer and айне. 

my To WF Tablets 


a 





This Engish language page on 
the website of Samsung, the 
company that makes this tablet, 
has helpful content. Although 
there are some helpful images on 
the page, very few Korean users 
Would be abie to read this page. 





Copyright 2021 


160 





Оп the other hand, there may be queries where the needs of most users would actually be satisfied by English language 
results, even if the query itself is typed in the task language. 


For example, for queries about global businesses and organizations, users may expect or want to visit the English 
language version of the business/organization's official website in some locales. Similarly, for queries seeking technical 
information such as manufacturer part numbers, product specs, scientific or chemical formulas, etc. the answer to the 
query may be typically expressed in the English language in some locales. 


For these queries, users may expect or want to see English results in order to satisfy their need. Please use your 
judgment and knowledge of your locale to determine the appropriate rating. 





Query and User intent 


Result Block and LP, Needs Met Rating 


Explanation 





Query: fer teer] 
Locale: Hindi (IN) 


User Location: Delhi, India 


User Intent: dd ече is 
"Harvard website” п Hindi. Users 
want to go to the official website 
of Harvard University. which is а 
'weltkmown research university in 
Cambridge, Massachusetts. 


Harvard University 
wwe harvard edu 


Harvard University в devoted to excellence in teaching. 
learning, and research, and to developing leaders in mary 
disciplines who make a difference globally Harvard 
university is made up of 11 principal academic 
Admissions & Aid Schools Faculty Harvard College 


This is the official website of 
Harvard University. Even though 
the query was typed in Hindi, this 
Engish website s clear what 
users are looking for. Users 
would be fully sat by this 
result. 





Query: AIA 182! 
os Saspi] 





^n 


Locale: Korean (KR) 


User Location: Seoul, Korea 


User Intent: This Korean query 
can be translated as "recovering 
105 on Cisco Catalyst switches”. 
The user wants o find out how to 
restore the IOS operating system 
оп a Cisco Catalyst switch, which 
is a эрес brandimodel of 
‘computer networking device. 











Cisco Catalyst A917 (2900, 3500, 3750) 105 525} 
2|:Ho жал 
mblog Sovercom 220182670678 


2016.6, 20, -Cisco 10S Software, C3560 Software (C3560- 
IPSERVICESAB-M) Version 12 (5318... Technical 
Support http:wwu cisco contechsupport. 








“This article comes from a blog 
‘that contains lots of technical 
information and troubleshooting 
ips about IT operations and 
programming. The landing page 
provides step-by-step instructions 
n Korean. along wih the exact 
commands in English that users 
would need to type into a console. 
"n order to recover the operating 
system for the queried device, 
‘This tutorial would be very перы 
and easy to follow for Korean 
users who are familiar ий this 
уре of networking device. 
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Finally, in some locales, English is one of the official languages or a commonly spoken language. In these locales, 
English websites are easy to use and could be helpful, depending on the query. 


For example, the Singapore govemment recognizes four official languages: English, Malay, Chinese, and Tamil, but 
English is the first and most dominant language in Singapore. 





‘Query and User Intent Result Block and LP, Needs Met Rating Explanation 





Barack Obama - Wikipedia. 
enm wikipedia orsjwk/Barack Obama 
Locale: English (5б) Mobley» Barack Hussain Obama Ii 
an Amarican poiiician who served as tho 44th This Wikipedia page in Engish 
President of he United Stats from 2009 lo peer Ohama ош he vory 
(2017. не is he frst African American to pullo users in Singapore. 


Query: [barack obama] 





User Location: Singapore, 
Singapore 


User Intent: Find information 
about Barack Obama. pe E 

















24.0 Rating Dictionary and Encyclopedia Results for Different Queries 


When assigning Needs Met ratings for dictionary and encyclopedia results, careful attention must be paid to the user 
intent. Like all results the helpfulness of dictionary and encyclopedia results depend on the query and user intent. 


Dictionary and encyclopedia results may be topically relevant for many searches, but often these results аге not helpful for 
соттоп words that most people in your rating locale already understand. Reserve high Needs Met ratings for dictionary 
and encyclopedia results when the user intent for the query is likely "what is it" or "what does it mean” and the result is 
helpful for users seeking that type of information. 


Important: Think very carefully about the helpfulness of dictionary and encyclopedia results for ordinary words and 
common items. If few users would benefit from a dictionary or encyclopedia result for a common word, a Slightly Meets 
rating may be appropriate. If very few or no users would benefit, then Fails to Meet is appropriate. 





Query and User Intent Result Block and LP, Needs Met Rating Explanation 





СТЕ 

Obsequious | Definition of Obsequious by —— Sn 
User Location: Miwaukee, Merriam-Webster tid n socondatyschoo апа 
== Obsequious defitons- marked by or esli oling a don و ا ا‎ 
Usar intant: Users are probably | fama attentiveness How o use bsequlous na RST 
бооп for he defintion ота, |  emence Follow Along ith he tionary rasa hot. 


vocabulary word. 
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‘Query and User Intent 


Result Block and LP, Needs Met Rating 


Explanation 





‘Query: [rainbow] 


User Location: Lincoln, 
Nebraska 


User Intent: This is a broad 
query. Users could be looking for 


many things, including images of 
rainbows, brands or юса! 


storesiorganizatins with 
rainbow" in their name, etc. 


Rainbow | Definition of Rainbow by Merriam: 
Webster 


Rainbow definition ts-an arc or crie that exhibns т 
concentre banda the colors of he spectrum and hat ie 
formed opposite tne sun by the refraction nd. 


Most users probably already 
know what a rainbow is and 
wouid not ba looking for a 
defimton. Even f the user needs 
heip understanding the word, 
images or a simple description 
wouid be more helpful than this 
dictionary result. 





Query: [cafeteria] 
User Location: Eugene, Oragon 


User Intent: Most 
English-speaking users in tha US 
are familar with cafeterias. Users 
aro most Ikely looking for a place 
ова. 


W Wied wt Caleta 


Cafeteria -Wikipedia 


A cafeteria, sometimes called a 
canteen outside the U.S, в a ype of 
food service location in which here is 
rte or no wating staff table service, 
whether a restaurant or within an 
institution such as a large office 
building or school: а school dining locaton is aio referred 
ta as a dining hall or lunchroom fin. 





Histry Othernames College cafeteria 





Most users are looking for a place 
to eat rather than general 
information about cafeterias 

Even though this is an in-depth 
[encyclopedia arie that matches 
the topic of the query, it would be 
helpful for few users. 








Query: [history of ATM machines] 


User Location: Birmingham, 
‘Alabama 


User Intent: Find information 
About the history of ATM 
‘machines (eg. the inventar 
when it was inverted, its 
‘evolution, ete). 





ле Марада wii Atene tle 


Automated teller machine - Wikipedia. 


lump o History On September 2. 1969, Chemical Bank 
Installed the frst АТМ in he U.S. at ns branch n Rockvlle 
ente, New York. The fist ATM were designed to 
dispense a fired amount of cash when a user inserted a 
specialy coded card. 


Location Financial networks Global use || Harbvan 








The user wants o learn more 
information about he history of 
АТМ machines, This 
[encyclopedia atte includes an 
‘extensive section оп ATM 
machine history. This result 
would be very helpful for users 
locking to ieam more about he 
history of ATM machines. 
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‘Query and User intent 


Result Block and LP, Needs Met Rating 


Explanation 





Query: [atm near me] 


User Location: Livingston. 
Wyoming 


User Intent: Find ATMs near the 
user locato. 


ле kpeda wii Atene tle 


Automated teller machina - Wikipedia. 


lump o History On September 2. 1969, Chemical Bank 
installed the frst АТМ in he U.S. at ns branch in Rockvlle 
ente, New York. The fist ATMs were designed to 
dispense a fired amount af cash when a user Inserted a 
specially coded card. 


Location  Fmancalneworks = Global use | | Hardwan 


The user intent is to find a nearby 
ATM machine. This encyclopedia 
rici fais to meet he need of 
‘the user because ithas no 
information about nearby ATM 
locations 








Query: [atm 
User Location: El Paso, Texas 


User Intent: Most users are 
probably looking for ATMs nearby. 
Fewer users would be looking ог 
general information about ATMS 
stor, how they work, ot. 





ле Марада wii Atene tle 


Automated teller machine - Wikipedia 


lump to History: Оп September 2, 1969, Chemical Bank 
installed the first ATM inthe US. at hs branch in Rockville 
Centre, New York. The frst ATMS were designed to 
dispense a feed amount of cash when a user inserted a 
specially coded card. 


Location  Fnanclal networks Global use | | Hardwart 





This encyclopedia article about 
ATMs is a minor interpretation of 
Ihe query. This result would be 
helpful far fewer users: most 
users would be looking far 
locations о! ATMs nearby. 
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Appendix 1: Using the Evaluation Platform 


250 Overview 


The Evaluation Platform is the system you will use to acquire and rate tasks. You can access the Evaluation Platform by 
going to this link: http-Jwwwraterhub.com/avaluationirater 


Аз а Search Quality Rater, you will work on many different types of rating tasks. This section provides guidance on how 
to use certain features of the Evaluation Platform that you will encounter frequently in Page Quality (РО) and Needs Met 
(NM) rating tasks. Some of these concepts may apply to other types of rating tasks as well. 

Important: Many tasks that you receive wil display project-specific instructions on the task page. You should always 
follow these project-specific instructions when they are provided. 


26.0 Acquiring Tasks 


Here is a simplified screenshot of the task acquisition page on the Evaluation Platform. Please note that changes аге 
sometimes made to the appearance of this page, so it may not look exactly as it appears below. 





Desktop Tasks 


A few notes about how to acquire tasks from this page: 


= rater@gmail.com: Make sure you are logged in to the account you used to sign up for the rating program. 





* Acquire if available: Click this button to acquire a new task. 


» Headphones or speakers required: This label indicates that the task requires headphones or speakers (for 
‘example, to listen to an audio file). 


Please note you may see other types of labels next to the button on the task acquisition page. Sometimes there 
wil be only one button, while other mes you may see more than one button displayed to distinguish tasks with 
different requirements. When there is more than one displayed, you may choose the one you want to acquire. 

27.0 Rating Tasks Using the Rating Interface 

Here is а general outline of the steps you should follow upon acquiring a task 

1. Read the instructions: Pay close attention to any project-specific instructions and examples if provided. 
2. Fill out the task: Assign ratings to result blocks, provide comments, answer questions, etc. (as applicable). 


3, Check your work: Verify that you entered your ratings as intended. Additionally, make sure that you provided а 
response to all required questions. 





4. Submit the task: Click the “Submit” button at the end of the page to submit the task and continue rating. 
Alternatively, click "Submit and Stop Rating” to submit the task and return to the Evaluation Platform homepage. 


Copyright 2021 165 


Additionally, here a 





a few general things to keep in mind when rating tasks: 

‘© Result blocks are sometimes slow to load: Before assigning any ratings, please scan the task page to make 
sure that all result blocks have loaded. Please note that result blacks may be slow to load and/or that the task 
раде may need reloading if there are empty result blocks. If the result blocks fail to load after refreshing a few 
times, please release the task by following the instructions in Section 28.0 below. 


‘= Sometimes a side will not have any result blocks: If one side displays the message "This side did not 
generate any results,” please do not release the task. 


* бо not refrosh the task page after assigning ratings: If you refresh the page after assigning ratings, you will 
lose them and they wil have to be entered again. 
28.0 Releasing Tasks 


Most raters have difficulty rating some tasks now and then. Some queries are about highly technical topics (e.g. 
‘computer science or physics) or involve very specialized areas of interest (e.g., gaming or torrents). 


You will report problems and release tasks using the "Report a Problem / Release this Task” button in the rating interface. 
Here is what the “Report a Problem / Release this Task button looks like: 


меп Relea 





ms 





Clicking the red button displays a menu with several reporting and release options: 





Rapor a Problem Release this Task E 


ett ишш nt ped ie pate a URL 











na -Ataman атаны 











Please note that changes are sometimes made to this menu, so it may not look exactly as it appears above. 
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їп general, you should choose whichever option bast describes your reason for releasing the task. If you are unsure, 
please select the closest option, or contact your employericompany for further guidance. 


Please leave a comment explaining the release when it makes sense to provide additional information, for example, when 
you are releasing for a "technical problem." 


Here are some examples for the "lack expertise" and "technical problem release options. 

















Lack expertise: These queries require specialized knowledge and may be dificult for some raters. 
Query [General Query Topic 

[cortisol secretion shows a blank pattem of secretion] | Scientific query 

Iminecraft gun and vehicles 1.1 mod installer [Gaming query 

[python pandas index like matrix] Technical query 














Technical problem: These result blocks have question marks and other characters where text should be. 











LT or Finds Z ultimate 1 


Note: Based on the number andor type of tasks that you release, you may be asked to provide additional details about 
the reason for some of the releases. 
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29.0 Understanding the User Location on the 


Task Page 


The location of the user should be inferred from the map provided. The map can help clarify where the user was. 


physically located when the query was issued, 


Оп the map, the user location can be precise (blue dol), approximate (blue rectangle or circle), or missing altogether if the 
user location is unknown. Here are some examples of what you might see at the top of the task: 





A precise user location 
is represented by a bue 


ot) 


Sometimes you wil see 
a shaded blue circle 
around the blue dol. The 
Blue dotis he user's 
most kay location, but 
itis possible they are 
located anywhere within 
tha larger shaded blue 
ragion. 





The user can be located 
anywhere within the 
lus rectangle on th 
map. 











The map includes а 
response pin, which is a 
marker for a 
Visiir-person result 
Моск in the task If 
"here are three 
visiti-person result 
blocks in the task there 
shouldbe three 
response pins on the 
тар. 














30.0 Reporting Duplicate Results in Tasks 


In some rating tasks, you may be asked to determine whether any results are Duplicates (dupes). This section provides 
Guidance and examples of how to report duplicate results In Needs Met (NM) rating tasks. 


30.  Pre-dentified Duplicates 


‘Some duplicate results are automatically detected and will be 
"Samo as... right below the slider and above the result block. 


This is what these pre-identified dupes look like: 


pre-identified for you. They will be annotated by the text 





Left Side 


[Right Side 




















THE 10 BEST Chats Hota wth a Pool o 2019 one 











Please note: You cannot unselect dupes that have been automatically detected and pre-identified 
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302  Rater-dentified Duplicates 


You are also asked to help identify duplicate results that have not been automatically detected. Please mark two results 
as dupes if they have essentially the same content on the main landing page AND you would nat want a search engine to 
retum both results for the query. 


Please note that in Needs Met rating and other query-based tasks, dupe identification is query-dependont. 


* Specific queries: For queries where the user is looking for a specific piece of content (such as queres looking for 
song lyrics, queries looking for a specific article, etc.) obtaining that piece of content пот different sites could 
bo helpful for users to vert the information, so they should not be rated as dupes. 


‘© Broad queries: if the query is broad, then returning the same piece of content is not what the user is looking for, 
and hence those results should be flagged as dupes. Results may be considered dupes even if they have minor 
content differences on the page (such as different ads, images, or related links). 


Please identify dupes both within the same side and across sides. Even for cross-side results, you should still ask 
yourself the question, "Would users want to see bath results if hey were returned by the same search engine?" 


Noto: If two result blocks have very different types of content or very diferent appearances, do not mark them as 
duplicates even if they have the same landing page URL. For example, these two blocks should not be marked as 
duplicates of each other. 








Dishdash Dishdash 
mo" wow atshaash com 
ы Motie-iensl - Catering Please vst our 


Dishdash famiy of restaurants. ador pag 
s ana pg - yelp.png ' FBTOOp« png 
nay instagram png 

Dinner Lunch - Our Story - Order 
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ate Results 





When you notice that the results in two or more result blocks are duplicates, please click on the Select Dupe Ink at the 
top of one of the results. The current result (the result you clicked on) will be highlighted in red at the top of the block. 





= p 





You can then check results that are duplicates of the current result, and the checked results wil be highlighted in yellow. 
You must select the "Dupe of..." button. 





The duplicate results that you checked will be annotated by "Dupe of..." text at the top of the block. After selecting ай 
dupes, click on the finish selecting dupes link to retum to the normal rating mode. The links name will change back to 
Select Dupes, and you wil be able to report other sets of dupes, if there are any. If you change your mind, you сап 
always uncheck a result. 


Dupes 

QUERY: [choosing and installing a motorcycle battery] 

URL 1: htip:/iwww.caimag,com/wordpress/20 10/03/06/motoreycle-battery-how.to-choose- install 

URL 2: htp:/iwww articlesbase,com/motorcycles-articles/choosing-and.instaling-a-motoreycle-battery.47798 him| 
Reason: Both of these results display the same article (which also appears on many other pages on the web). The only 
real difference between the landing pages are the Ads displayed around the article. The query is broad enough that users 
would not benefit by search engines retuming more than опе of these results, 


Dupes 

QUERY: {jason castro] 

URL 1: htp;wvwmyspace.comijasoncastromusic. 

URL 2: htip-iwvew.myspace.comijasoncastromusic?MyToken-S03599bf.01cf-427-bdf-d63920c107/9 

Reason: These two results have the same landing page, even though the URLs are different. Users would not benefit by 
search engines returning bath results. 


Not Dupes 

QUERY: [material girt lyrics] 

URL 1: htp;wveslvricstreak.com/m/madonna/material« qid. 2008895 him 

URL 2: htp:eveslvricstO7 ,com/Madonna"420L yrics/Material®% 20Girh20Lyrics, htm! 

Reason: Even though both pages display the lyrics to the song "Material Girl users would probably want to have the 
option to visit bath pages so that they could verify the accuracy of the lyrics. Users could benefit by search engines 
retuming more than one page with the lyrics to the song. 
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310 Simplified Needs Met Tasks 


‘Some Needs Met rating tasks may not require ratings for all blocks. In these blocks, the Needs Met and Page Quality 


rating scales will be greyed out and annotated with the text "No Rating Required" 


The greyed-out results (sometimes referred to as "contextual results) are shown to help you understand the query and 


Inform your ratings. You can click on them when you fee! like they will be helpful in determining your ratings. 


Here is an example: 





























m 


Appendix 2: Guideline Change Log 


This guideline is updated regularly to remain relevant aver time. Changes from the past two years (October 2019 — 
October 2021) аге summarized below. 





Dato Published 


‘Summary of Changes 





October 2021 


Expanded the definition of the YMYL subcategory "Groups of people" 
Refreshed guidance on how to research reputation information for websites and 
content creators 

Restructured and updated ‘Lowest Page Quality section; reorganized and 
refreshed examples to reflect new structure 

Simplified the definition of 'Upsetting-Offensive' to remove redundancy with 
Lowest Page Quality section 

Minor changes throughout (updated screenshots and URLs, wording, and 
‘examples for consistency; removed outdated examples; fixed typos; etc.) 





October 2020 


‘Added note to clarity that ratings do not directly impact order of search results 
Emphasized The Role of Examples in those Guidelines! as an independent 
section in the introduction 

‘Added clarification that Special Content Result Blocks may have links to landing 
pages; added illustrative example 

Updated guidance on how to rate pages with malware wamings and when to 
assign the Did Not Load flag: added illustrative examples 

Changed the order of Rating Flags section and Relationship between Page Quality 
апа Needs Met section for clarity 

‘Added "Rating Dictionary and Encyclopedia Results for Different Queries’ 
Emphasizes the importance of understanding the user intent and query for Needs 
Met rating; added illustrative examples 

Minor changes throughout (updated examples and explanations for consistency; 
simplified language regarding raters representing people in their locale; fixed 

typos; etc.) 








December 2019 








‘Added Introduction to Search Quality Rating’ Describes the overall search user. 
experience and the purpose of search quality rating 

‘Added definiions and clarifications to explain key search concepts 

Minor changes throughout (removed obsolete QR code; revised language for 
consistency across sections; fixed typos; etc.) 
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